Interactive data view and command system

ABSTRACT

A method and information system for capturing signals, for processing signals, and for providing signals at least partially based on, or bearing correlation to, the captured signals is disclosed. The information system includes a signal input unit, a wireless communication unit, and an output unit. The signal input unit (preferably an optical signal unit) is constructed and is positionable to capture signals associated with an eye. The output unit is constructed to provide information based on the captured signals or to provide information as a function of the captured signals or in correlation with the captured signals.

FIELD OF THE INVENTION

[0001] The present invention relates to an interactive data view andcommand system including a wearable signal input unit providing highlevel of wearer comfort, performance and flexibility.

BACKGROUND OF THE INVENTION

[0002] Recording, processing and reproduction of images grows with anincreasing demand for information and their clear graphic visualization.Electronic image processing also involves processing of images taken bycameras, scanning systems and sensors in the visible light spectrum aswell as in other sections of the electromagnetic spectrum such as theinfrared, the radio frequency and the X-ray frequency range. Afterelectronic processing, many images are reproduced either as individualimages or as moving images on an image reproduction screen such as adisplay for presenting the information to the eye.

[0003] There are special image contents easier recognizable byelectronic image processing using, for example, local frequencyfiltering, margin sharpness increasing, image data compression, imagecorrelation, dynamic reductions and false color-coding. On the otherhand, other techniques are concerned with the superposition orsubtraction of auxiliary images taken from different spectral ranges orwith the superimposing of stored plans, maps, and drawings onto theoriginal image.

[0004] By applying image processing, the information content of theactual, direct image can be intentionally increased or reduced. Imageprocessing is used in a wide range from increasing the image contrast toblending-in of additional information, marking of details, andhighlighting dangers. In many of these applications, it isdisadvantageous that the electronic camera is a “second eye system”separate from the human eye. This disadvantage is due to the fact thatthe images are seen from another recording location and thatadditionally, the pictures on the image screen are presented at anotherobservation location than the eye. Thus, the human eye must constantlychange between the direct observation and the indirect observation whiletaking into account different observation angles, different imagedetails, and different size ratios which leads to physical impairmentsand delays when decisions must be made.

[0005] These problems may be solved by the “head-up-display” (HUD)technique used in the piloting of combat aircraft, in that importantinformation such as instrument displays and target data are inserted orfade-in into the open spectacles of the pilot's helmet and thus into thevisual field of the pilot. This technique is also used experimentally inthe automobile industry for displaying of instrument readings on thewindshield so that the driver is not distracted from viewing the road byviewing the instrument panel.

[0006] The “virtual reality” or “cyber space” techniques use closedspectacles (i.e. glasses) where the outside view is blocked andthree-dimensional full images are projected into the eye with virtualreality. These virtual reality images are then modified in response tobody motions such as locomotion, movement of an arm, a finger, or headand eye movements.

[0007] There are other HUD techniques and image detection and projectiontechniques described in the PCT Application PCT/EP97/04188 (published asWO98/05992) and U.S. Pat. No. 6,227,667. The apparatus described inthese documents capture (or detect) the retinal reflex image and alsoenable a superimposition of supplementary images in the eye.

[0008] There is still a need for an information system capable ofproviding interactive data view and command applications for use invarious environments.

SUMMARY OF THE INVENTION

[0009] The present invention is a method and information system forcapturing signals, for processing signals, and for providing signals atleast partially based on, or bearing correlation to, the capturedsignals. The information system includes a signal input unit, a wirelesscommunication unit, and an output unit. The signal input unit(preferably an optical signal unit) is constructed and is positionableto capture signals associated with an eye. The output unit isconstructed to provide information based on the captured signals or toprovide information as a function of the captured signals or incorrelation with the captured signals.

[0010] The signal input unit may be a non-optical signal unit forcapturing non-optical signals (e.g., acoustic data, electric data,magnetic data), or an optical signal unit for capturing optical data(e.g., ultraviolet, visible or infra-red). The optical signal unit cancapture data by scanning or large image detection using light sensitivearrays like CCDs. In some embodiments, the signal input unit may beintegrated with the output unit, and may also include an informationunit.

[0011] Preferably, the optical signal unit and possibly other units areconstructed in the form on spectacles or goggles worn by a user, asdescribed in the PCT Application PCT/EP97/04188 (published asWO98/05992) and U.S. Pat. No. 6,227,667; U.S. application Ser. No.09/462,440; or PCT Applications PCT/EP00/09840 PCT/EP00/09841 andPCT/EP00/09843, filed on Nov. 6, 2000; the content of these applicationsis explicitly incorporated by reference into this application as iffully provided herein. The various embodiments and units described inthe above patent documents have been further improved to createeconomical, user-friendly, comfortable, high performance devices. Inparticular, adjustments and improvements of the aforementioned opticalunits provide a novel manner of projecting images onto or in front ofthe user's eye, including the retina and other surfaces. The presentoptical signal unit and output unit are not limited to scan-like captureof the retinal reflex image or projection on the retina, but utilizeother properties and function of the eye and human perception.

[0012] Since the apparatuses and systems described in the aforementionedapplications are preferably embodied in the shape of spectacles, theywill also be designated hereinafter, for the sake of simplicity, as aspectacle system, but this designation does not and should not imply anylimitation. The optical apparatus that scans the retina and displayscorresponding image modifying or image improving optical informationonto the retina are optimally used.

[0013] The signal input unit may be an optical signal unit isconstructed to capture visible signals, infrared signals or ultravioletsignals. The output unit may be constructed to provide the informationon a display by projection or using a graphical user interface. Theoutput unit may be constructed to project the information. For example,the output unit is constructed to project at least part of theinformation onto the retina of the eye, or the output unit isconstructed to project at least part of the information onto a two orthree dimensional surface remotely located from the eye. The output unitmay be constructed to provide the information in an audible format. Theoutput unit is constructed to provide the information in a formatcausing sensory skin stimulation, stimulating tactile organs, or visual,audible, smellable, or tastable senses.

[0014] According to another aspect, an information system includes anoptical signal unit, a wireless communication unit, and an output unit.The optical signal unit is constructed and positioned to capture signalsassociated with an eye. The output unit is interfaced with the wirelesscommunication unit and is constructed and arranged to provideinformation using a correlation unit. The correlation unit isconstructed to find suitable relationship between the captured signalsand additional data.

[0015] Preferred embodiments of this aspect include one or more of thefollowing features:

[0016] The information system may also include an information unit. Theinformation unit may include various databases, processors, sensors, aninformation network connection, an evaluation unit, and other devicesproviding additional data to the information system.

[0017] The correlation unit is constructed to determine a presentationrelationship of the captured signals and the additional data. Thecorrelation unit is constructed to determine the presentationrelationship between the captured signals and the additional data interms of location of the additional data with respect to the captureddata. Alternatively, the correlation unit is constructed to determinethe presentation relationship between the captured signals and theadditional data in terms of presentation timing of the additional datawith respect to the captured data.

[0018] The correlation unit may be constructed to determine thepresentation relationship between the captured signals and theadditional data in terms of relative color display of the additionaldata with respect to the captured data. Alternatively, the correlationunit is constructed to determine the presentation relationship betweenthe captured signals and the additional data in terms of relative sizeof the additional data with respect to the captured data.

[0019] In short, the correlation unit performs the correlation functionthat may be “one-to-one” overlap of a scanned and a projected image, ormay completely differ from the “one-to-one” overlap, or may includedifferent type of data. For example, the input signal unit may include amicrophone to the external system, in particular a directionalmicrophone, which is directed as a function of the head position or thegaze direction to allows a further sensory dimension to be realized.

[0020] The correlation unit may direct projection of a correlatedfield-of-view image of an individual (e.g., a fireman) or a mosaic ofseveral images into the eye or on a projection surface of anotherindividual (e.g., a fire chief overseeing the operation). Thecorrelation unit enables, for example, the projected image of the entirescene (field-of-vision) to be partially overlapped by data symbolizingthe position and severity of the sighted or otherwise captured (ordetected) event (e.g., fire location and temperature colored inaccordance with the strength of the fire, which could be blended intoeach fireman's view). Alternatively, a disruptive effect of the naturalfield of view not necessarily covered up by the projection ofinformation can be reduced via a complementary-color “wiping out” inwhich complementary-colored pixels are determined on the basis of thelight captured from the field of view whose correlated projection ontothe respectively associated regions of the retina make the naturalbackground appear white on account of the addition of colors. Thevirtual position of the fixation fixed, e.g., via a fixation with theeye in conjunction with a blinking of the eyes (or a pressing of abutton or even automatically), for example, by an image-processingevaluation of the field of vision that determines an area of the fieldof vision having as little content as possible.

[0021] According to yet another aspect, an information system includesan optical signal unit, an information unit, a wireless communicationunit, and an output unit. The optical signal unit is constructed andpositioned to capture signals reflected back from at least one eyecomprising the retina. The field-of-view capturing unit is constructedand arranged to capture light from a field of view associated with theretina without capturing a retinal reflex image. The output unit isconstructed to provide information, at least partially obtained via thecommunication unit, in cooperation with the information unit as afunction of the captured light and in correlation with the capturedsignals.

[0022] According to another aspect, an information system includes anoptical signal unit, an information unit, a wireless communication unit,and an output unit. The optical signal unit is constructed andpositioned to capture signals reflected back from at least one eyecomprising the retina. The optical signal unit comprises a scanningdetection unit constructed to at least partially capture a retinalreflex image of the retina. The output unit is constructed and arrangedto provide information, at least partially obtained via thecommunication unit, in cooperation with the information unit as afunction of the captured signals.

[0023] The output unit may be cooperatively constructed and arrangedwith the optical signal unit to enable projection of information ontothe retina. Alternatively, the output unit is not constructed to projectinformation onto the retina.

[0024] According to yet another aspect, an information system includesan optical signal unit, an information unit, a wireless communicationunit, and an output unit. The optical signal unit is constructed andpositioned to capture signals reflected back from at least one eyecomprising the retina. The optical signal unit comprises a scanningdetection unit constructed to at least partially capture a retinalreflex image of the retina during a first scanning operation andcarrying out a less comprehensive capture of the retinal reflex imageduring a later scanning operation. The output unit is constructed andarranged to provide information, at least partially obtained via thecommunication unit, in cooperation with the information unit as afunction of the captured signals. The output unit comprises a scanningprojection device constructed to project at least part of theinformation onto the retina.

[0025] According to yet another aspect, an information system includes asignal input unit, an information unit, a wireless communication unit,and an output unit. The signal input unit is constructed not to captureany signals reflected back from the retina. The output unit isconstructed and arranged to provide information, at least partiallyobtained via the communication unit in cooperation with the informationunit as a function of the captured signals. The output unit comprises ascanning projection device constructed and arranged to project at leastpart of the information onto the retina.

[0026] According to this aspect, the signal input unit may be an opticalsignal unit constructed and positioned to capture signals reflected backfrom at least one eye but not from the retina.

[0027] According to yet another aspect, an information system includes asignal input unit, an information unit, a wireless communication unit,and an output unit. The signal input unit is constructed and positionedto capture at least two types of signals wherein one or both signals maybe reflected back from the eye. The output unit constructed and arrangedto provide information, at least partially obtained and/or provided viathe communication unit, in cooperation with the information apparatus asa function of the captured signals. The output unit comprises a scanningprojection device constructed to project at least part of theinformation onto the retina of the eye.

[0028] Preferred embodiments of the above aspects include one or more ofthe following features:

[0029] The information system includes a spherical or spherical-actingreflection layer operably positionable at a location immediatelyanterior and substantially confocal to the eye, and the optical signalunit is constructed to capture optical field-of-vision signals reflectedoff the spherical or spherical-acting reflection layer.

[0030] The field-of-view capturing unit is constructed to capturevisible light from a field of view associated with the retina withoutcapturing a retinal reflex image thereof. The output unit is suitablefor providing the information in correlation with the captured visiblelight.

[0031] The information unit comprises an evaluation module constructedto obtain image information with regard to the field of view from thecaptured visible light.

[0032] The output unit comprises a projection device constructed toproject the image information onto the retina in correlation with thecaptured signals such that a naturally perceived field of view andprojected image information are perceived as a unitary image by theretina. The scanning and projection may utilize separate beam paths.

[0033] The information system provides a function that encompasses atemporal or spatial correlation between the provision of information andthe captured visible light. This function may also encompasses patternrecognition that yields at least one information key, and theinformation keys serve for an information query based on the informationapparatus.

[0034] The optical signal unit includes a scanning device that recordsan at least partial capture of a retinal reflex image of the retina in afirst scan operation and carries out a less comprehensive capture of theretinal reflex image in a later scan operation. The optical signal unitmay capture a retinal reflex image of the retina only partially or notat all.

[0035] The field-of-view capturing apparatus includes a spherical orspherical-acting reflection layer suitable for deflecting a portion of alight directed at the eye into a sensor apparatus for capture. Thefield-of-view capturing unit or the optical signal unit may be suitablefor at least partially capturing a corneal reflex image of the eye. Theoptical signal unit or the field-of-view capturing unit may befabricated as portable units. The information unit may include adatabank (including various storage devices), a sensor system, aninformation network connection and/or an evaluation unit. Theinformation unit may be fabricated as a portable unit.

[0036] In accordance with one aspect of the optical input unit, theoptical apparatus is wearable by an operator and capable of scanning theretina up to frequencies of 100 Hz and an image falling onto the eye, inparticular a retinal reflex image. The optical input unit is a part ofan interactive data view and command system, with the particularadvantage that, for operating the system, a least possible effort on thepart of the operator is necessary. The information to be queried can betransmitted to the operator with shortest possible delay either in theform of image signals and/or in the form of signals for controllingfurther information reproduction devices that can be operated e.g. on anacoustic or other sensory basis. A system thus results that isdistinguished by a highest measure of directness of the interactiveinformation exchange.

[0037] The data transmission unit may include a mobile communicationsystem is switched between the data view and command system and anexternal information source. In this manner, the field of application ofthe entire information system is additionally extended. The datatransmission unit can include a mobile telephone or a computer, e.g., alaptop or palmtop having a suitable data remote transmission interface.

[0038] The system can execute additional control commands that controlthe flow of information. These control commands can be based on opticalinformation signals that are output by the optical apparatus. Forexample, a menu bar is blended in to the operator into the field of viewby outputting image signals onto the retina via the optical apparatus,again with an image frequency of roughly up to 100 Hz. The selection ofa menu item is carried out either via a control mouse, or instead solelyvia the focusing of this selected menu item and with the aid of aselection signal. The selection signal can be a button, voice command oran eye blink signal. The focusing can be determined optically since theselected menu item is situated in the center of the fovea centralis,which can be determined, in parallel, while the scan of the retina/ofthe image falling onto the eye is running.

[0039] The signal input unit may include a microphone and a speechanalysis unit.

[0040] The operator's additional control commands can be based onacoustic information signals that are output from a speech input unit.Since, the signal input unit may include, an optical apparatus is usedthat cyclically scans the operator's retina, and the system can beemployed to carry out an active operator recognition in a highlyeconomical fashion. It is thus possible, on the one hand, toautomatically protect the system/system operations from unauthorized useand, on the other hand, to carry out an automatic personalized set-up ofthe system to the operator wearing the system. For this purpose, it issolely necessary to temporarily store the retina structure captured bythe optical apparatus in the form of a data record in the signalprocessing apparatus and to compare it to a person-specific data recordalready stored.

[0041] Due to the compactness and the short signal paths from theoptical apparatus to the eye, on the one hand, and to the communicationapparatus on the other hand, application of the system is quiteversatile including the use in the realm of medicine, in particular inthe field of ophthalmology, as both a therapeutic and analytic device.

[0042] Additional applications are described in the PCT ApplicationsPCT/EP00/09840, PCT/EP00/09841 and PCT/EP00/09843, which areincorporated by reference.

[0043] The present information system includes numerous embodiments thatprovide one or more of the following features:

[0044] at least partially captures a cornea reflex image of the eye;

[0045] directs a part of the light falling onto the eye into a sensorapparatus by using the spherical or spherical-acting reflection layer;

[0046] determines the retinal image via the degree of oxidation of theretinal cones and/or the retinal rods;

[0047] carries out solely a partial capture of a retinal reflex image;and/or

[0048] comprises a field-of-view capturing apparatus that capturesvisible light from the naturally perceived field of view withoutcapturing a retinal reflex image.

[0049] At the same time, the present system in accordance with thedescribed embodiments can provide:

[0050] (a) information as a function of a naturally perceived field ofview of a person;

[0051] (b) information as a function of signals captured from an eye,yet does not project these into the eye from which the signals werecaptured; or

[0052] (c) information as a function of signals captured from an eye,wherein the information is at least partially projected into the eye,the signals, however, are not captured in the manner known from the PCTApplication PCT/EP97/04188 (published as WO98/05992) or U.S. Pat. No.6,227,667.

[0053] The signal input and output units, including the describedspectacle systems, are based on a combination of several of theaforementioned fundamental concepts, the result of which is a naturalinterrelationship of the associated three applications. The presentinformation systems allow additional information to be provided to theoperator that goes beyond our personal knowledge and sensoryimpressions. Examples of this include searching for an electrical cableunder the plaster in a wall, navigating in an unfamiliar city,collecting wild mushrooms, and inspecting a possibly dangerous objectusing a remote-controlled robot. The present information system canprovide information to a person or another system.

[0054] The strong dependency of a person having vision on their visualsenses clearly contributes to the difficulty of providing additionalinformation. Indeed, the fact that people having vision primarily usetheir eyes makes it necessary, in many cases, to either input thesupplementary information via the eyes or to determine the supplementaryinformation based on the information seen. In the case of input via theeyes, however, the orientation of the eyes must be taken into exactconsideration for correct “placement” of the input information and toavoid a “jittering” or “blurring” thereof. In addition, in many casesthe information should be made available without a controlled movementof the eyeballs; a car driver may have a map on his lap but would prefernot to have to look away from the street.

[0055] Due to their dependency on solid media, e.g. paper, CRT and LCDscreens, etc., prior visual information systems have not been in aposition to sufficiently fulfill the comfort needs of a person havingvision. Non-visual information systems previously lacked the correlationto that which is seen that is natural for people having vision.

[0056] Through the incorporation of the interactive data view andcommand system into an information system in accordance with thedependent claims, a system is provided whose presentation of informationfulfills the natural needs of person having vision in a previouslyunachieved manner. At the same time, the information system is improvedover the prior art with regard to its implementability and economy.

[0057] In its most general form, the information system in accordancewith the invention comprises a signal capturing apparatus that capturessignals reflected back from an eye comprising a retina, an informationapparatus and an output apparatus that provides, in cooperation with theinformation apparatus, information in correlation with the capturedsignals. Preferably, the information is provided as a function of thecaptured signals and/or as a function of visible light captured from thenaturally perceived field of view.

[0058] Preferably, one of the aforementioned spectacles in which ascanning detection apparatus at least partially captures a retinalreflex image of the retina serves as a signal capturing apparatus. Amodification of this detection apparatus that captures light reflectedon a cornea of the eye in lieu of the retinal reflex image isparticularly advantageous for infrared applications since the corneastrongly reflects light having a wavelength of roughly 1.1 μm. It isalso fundamentally possible to make correspondingly valuable statementsabout the image falling onto the retina by capturing the chemical changeof the rods and/or cones.

[0059] The present system uses the advantages of capturing the field ofview complementary to the capturing of signals reflected back from theeye. For the purpose of such a complementary capturing, thefield-of-view capturing apparatus and/or the information apparatus ofthe information system in accordance with the invention preferablycomprises a spherical or spherical-acting reflection layer that ispositioned essentially confocal to the eye that deflects a part of thelight directed onto the eye into a sensor apparatus for capture. Due tothe fact that the reflectivity of the reflective layer is several timeshigher than that of the retinal or corneal reflex, considerably morelight can be captured using equally sensitive photo-sensors. Moreover,correspondingly cheap photo-sensors could be used in the sensorapparatus. It can also be advantageous if the light falling onto the eyeis not solely, only partially or not at all captured via the retinalreflex.

[0060] Depending on the intended application, not all spatial regions ofthe field of view must be captured. In an application, for example, inwhich supplementary information with regard to an object upon which theeye has fixed its gaze is provided by an information system inaccordance with the invention, it could be sufficient to capture thelight falling onto the fovea and to subject it to a pattern detection orother type of analysis since an object upon which the eye has fixed itsgaze is typically imaged on the fovea which represents the area ofkeenest sight. Accordingly, capturing the light that falls onto thispart of the retina would possibly be sufficient to determine asufficient number of characterizing object features.

[0061] It is also sensible if only a limited spectral range of the lightfalling onto the eye is detected. If, for example, the infrared lightfalling onto an eye is detected, the orientation of the eye and/orvaluable information from the field of view can be determined, even atnight.

[0062] Accordingly, any limitations with regard to the capturing of thelight falling onto an eye can be meaningful. In particular, limitationsof the captured spectral range, the captured regions of the field ofview and/or the captured time spans of vision are applied as necessary.

[0063] For the purpose of redundant or stereoscopic image capture, thecorrespond apparatus of the information system in accordance with theinvention can be designed so as to capture the light falling ontoseveral eyes. Depending on the field of application, the eyes must notnecessarily belong to a single person. For example, it would be possibleto display the images perceived by the eyes of several firemen ontomonitors in a command center in addition to position and fire strengthinformation determined from an infrared spectral analysis of the images.

[0064] In the field of ophthalmology, a distinction is made between theterms “field of view” and “field of vision.” A field of view is the partof a space that can be seen with a stationary eye. A field of vision isthe region that can be seen with the eyes. Consequently, here, aselsewhere, the field of view is to be understood as the cause of thelight that naturally falls onto an eye.

[0065] The present system may use a field-of-view capturing unit (i.e.,field-of-view capturing apparatus) for capturing light from the field ofview associated with the eye of a particular quality, i.e. with asensitivity, a resolution, a sharpness, etc. This field-of-viewcapturing of visible, infrared or ultraviolet light far exceeds thenatural visual acuity of the eye. The field-of-view capturing unit isnot limited to capturing of the field of view, but may include partialor complete capturing of the field of vision that encompasses an atleast partial capturing of the field of view.

[0066] The field-of-view or field-of-vision capturing units capture highquality images that can also serve as a basis for an extrasensorypresentation of information. For example, field-of-view information canbe obtained from the captured light of the field of view and projectedonto the retina such that the image seen by the eye seems at leastpartially sharper, closer, wider angled or in some other mannerextrasensory.

[0067] The present system enables novel ways of correlating andpresenting data acquired, computed (manipulated), received via awireless communication unit, or from a storage device. Importantly, theprovision of information with the signals captured by the signalcapturing unit, the system treats the corresponding parts of thecaptured light during a processing that occurs in the course of theprovision of information as if they were reflex images captured from theeye, i.e., as if they were that which is truly seen. The informationsystem can also combine high quality field-of-view information directlyfrom the truly seen field-of-view information from the eye.

[0068] The correlation of the presentation of information with thecaptured signals reflected back from the eye can be carried out, forexample, by capturing several pixels of an ocular reflex image, e.g. acornea or retinal reflex image, that are brought into connection withcorresponding pixels from the captured field of view via an evaluationapparatus. A gaze direction of the eye determined via the capturedsignals can also serve to establish a correlation between the field ofview information obtained from the captured field of view and which istruly seen. As will be described herein below, the correlation can,however, also comprise projecting obtained field of view informationonto the retina in a correlated manner to that which is seen.

[0069] The information unit may include a data bank, a sensory unit, aninformation network connection, one or several processors, and/or anevaluation unit. The term “evaluation unit or apparatus” means any typeof evaluation apparatus and in particular image processing devices. Theinformation unit can include one or several sensory units collectingdata tactually, visually, audibly, smellably and/or tastably.

[0070] The sensory unit provides an extrasensory perception inconnection with the visible data. For example, when searching for anelectric cable, the sensory unit can include one or several magneticfield sensors capable of localizing metallic cables with regard to aknown coordinate system, e.g., the captured field of view. Imageprocessing software superimposes sensor signals from existing electricalcables and provides a supplementary image, as described inaforementioned patent applications, projected onto the image seen by theeye.

[0071] The information unit may also include other types of sensors usedas an information source, in particular when the sensor is activatedand/or queried on the basis of the captured light image. For example,during inspection of an integrated electronic circuit the position of aconductor on a manufactured chip could be computed after a directed gazeat that conductor on a circuit plan of the circuit and the pressing of abutton so that the current and voltage values of the conductor aredetermined using the non-contacting measuring device and presented tothe use via the spectacle unit.

[0072] The information unit may also include a data bank and aninformation network connection. For example, is an intra-company maildistribution system files include bar code stickers that uniquelyidentify the respective file. If a file is to be sent within thecompany, the sender enters e.g. the receiver's extension and a codedesignating the file using software that correspondingly stores thesedata in a data bank in one of the many known ways. When the file islater sorted, the identifying bar code is captured via the spectacleunit worn by a mail distribution employee, e.g., via a directed gaze anda push of a button, and recognized via a recognition apparatus orrecognition software. The data associated with the file relevant to maildistribution are retrieved from the data bank via a radio connection toan intra-company data network and these data are presented to the maildistribution employee, after pre-processing, if necessary, via asuitable output apparatus, e.g. as an announcement via headphones “Mr.Schmidt, finance department, building G, second floor, room 310.”

[0073] The presentation of information that complies with the needs of aperson having vision in a manner not previously achieved. This cancomprise providing the information to the person in a suitable manner,i.e. using one or more of the five senses. The information can, however,be presented in an arbitrary manner and does not require a particularaddressee. For example, the information can be provided to a furthersystem or radiated into the environment via an optical or acousticoutput apparatus. The claimed dependency between the provision ofinformation and the light image falling onto the eye guarantees that thecorrelation that a person having vision expects exists between theinformation provided and that which is seen.

[0074] This dependency is taken into consideration during thedetermination of the information, during the provision of theinformation or during both of these inherent processes. Examples forestablishing this dependency during the determination of the informationare given above. During the provision of information, this dependencycan be established, for example, by blending the information into theimage seen by projecting back into the eye in such a way that atemporal, spectral, spatial, contractual or other sensible correlationis established between the information and the image seen. Inparticular, the dependency can consist of the captured light image beingused to determine the position and orientation of the eyeball so that animage projected onto the eye for the sake of providing the informationappears to stand still during a motion of the eye, appears to move withthe eye during a motion of the eye or appears to move in accordance witha predetermined course during a motion of the eye. In particular, theeffect of the saccadic motion of the eye on these processes can be takeninto consideration and/or compensated.

[0075] The optical signal unit or the output unit (or the spectacleapparatus) can determine the position and orientation of at least oneeye quickly, accurately and at little expense, e.g. at a determiningrate of 100 Hz, a positional accuracy of several micrometers and using aportably constructed apparatus. By using the information during thedynamic evaluation of the orientation of the eye, the processing can becarried out so quickly that the accuracy is not impaired by the saccadicmotion of the eye. This is achieved by a signal-capturing unit, whichdoes not contact the eye and captures signals reflected back from theeye. Reflectable signals, e.g., sound or electromagnetic signals, allowa high frequency capturing such that the processing speed is primarilydetermined by an evaluation unit. The evaluation unit may have,depending on the application, very high processing speed, low powerconsumption and small system size.

[0076] The information system may itself serve as a reference coordinatesystem. However, the information system may solely represent anintermediate reference coordinate system in another reference coordinatesystem and that the relationship between the intermediate referencecoordinate system and the reference coordinate system is determined e.g.via the evaluation apparatus or another mechanism.

[0077] The signal input unit may include an optical signal unit thatcaptures light reflected back from the eye. Light is an excellent mediumfor transmitting the signals reflected back from the eye since thepresence of light is a prerequisite for the ability to use the eye.However, the ocular reflex signal information that results through thereflection on the eye is superimposed with field-of-view signalinformation transmitted by the light from the field of view. Thesediffering pieces of information, however, can be distinguished throughuse of known signal processing methods and can be sensibly used fordetermining the orientation of the eye. This is particularly true whenthe signal transmission medium is from a signal source belonging to theinformation system that applies a predetermined signal to the mediumprior to its reflection on the eye.

[0078] The signal input unit may also capture signals from other signaltransmission media. Components for generating and capturing sound waves,for example, are commercially available in various cost-efficient andcompact forms. Such components can also be implemented as integratedelements of an integrated circuit. Similar considerations apply to thenon-visible frequency range of electromagnetic waves.

[0079] The signal input unit may also capture signals from differentmedia or spectral ranges thus providing improved system characteristics.This is based on considerations that the evaluation unit takes overother system tasks in the case of underload and that the signalprocessing carried out by the evaluation unit depends strongly on theinformation content of the signal to be processed. The informationsystem can use signal capturing that only demands a little performancefrom the evaluation unit, but itself might not supply the basis forsufficient accuracy and to complement and/or calibrate thislow-processing signal capturing via the results of an accurate andprocessing-intensive, yet only intermittently executed signal capturingsuch that the necessary accuracy is achieved at any time.

[0080] The capturing of the retinal reflex in which the retinal reflexof natural or artificial light is intermittently or partial captured asthe signal reflected back from the eye has turned out to be useful. Afull capturing of the retinal reflex is both time consuming anddemanding on performance. On the other hand, a capturing of the retinalreflex is useful inasmuch as it allows the relationship of the perceivedfield of view to the retina to be directly determined. Indeed, as notedabove, a processing of the captured retinal reflex allows both retinalfeatures such as e.g. the fovea centralis or the blind spot as well asthe reflex image of the light falling onto the eye to be determined. Thenetwork of blood vessels present in the choroid coat also becomesvisible through appropriate processing of the retinal reflex image,which yields a very good basis for determining the orientation of theeyeball. If the retinal reflex is thus captured intermittently orpartially, the processing complexity can be reduced without sacrificingan exact determination of the relationship of the perceived field ofview to the retina. Naturally, the retinal features can be followedwithout capturing the retinal reflex. For example, the blood vessels ofthe choroid coat can be recognized via their radiation of heat that isvisible in the infrared range.

[0081] The above system may be used for the analysis of a patient'ssight, wherein a predetermined pattern or a predetermined distributionof patterns is generated on the retina or on selected regions of theretina using the projection unit. The system may use movement patternsand/or the noise fields and/or the spatial vision of a patient's eye,wherein random dot patterns are generated on the retina using theprojection unit for test purposes. The system may be used fordetermining anomalies in the motor response of the eyeball, wherein aunit for determining and monitoring the position and/or orientation ofthe eyeball is integrated into the system. The system may be used fordetecting parasympathetic/sympathetic efferences, wherein the motorresponse of the pupil is monitored and evaluated by means of a detectordevice. The system may also be used as one or more of the following: asynoptophor or synoptometer with no device convergence, a device fordetermining cyclodeviation, a phase difference haploscope, a device fordetecting phoria identical to the visual axis with different lines ofsight, for checking the function of the retina, making use of a sampleelectro-retinogram (ERG) and a correlation device, with which an imageplayed onto the retina can be brought into correlation with the ERGactually determined, for measuring the contrast sensitivity of apatient's sight, preferably as a function of the spatial frequency, forwhite-noise-field campimetry, for determining the extent and theposition of central field of vision defects (scotomae), as a VEP (visualenabling for precision surgery) device, or as an SLO (scanning laserophthalmoscope) device.

BRIEF DESCRIPTION OF THE DRAWINGS

[0082]FIG. 1 shows schematically an interactive data view and commandsystem in accordance with a first embodiment of the invention.

[0083]FIG. 2 is a detailed view of an eye in cross-section.

[0084]FIG. 3 shows schematically an embodiment of interactive spectaclesemployed in the invention in which an optical signal unit is provided inthe form of a scanning eye detection apparatus.

[0085]FIG. 4 shows schematically is an embodiment of interactivespectacles employed in the invention in which an output apparatus in theform of a scanning projection apparatus is provided.

[0086]FIG. 5A shows schematically interactive spectacles in accordancewith a fourth embodiment.

[0087]FIG. 5B is a detailed drawing of a combined signal capturing andprojection unit illustrated in FIG. 5.

[0088]FIG. 6A shows schematically interactive spectacles in accordancewith a fifth embodiment.

[0089]FIG. 6B is a detailed drawing of a combined signal capturing andprojection unit illustrated in FIG. 6A.

[0090]FIG. 7A shows schematically interactive spectacles in accordancewith a sixth embodiment.

[0091]FIG. 7B is a detailed drawing of a combined signal capturing andprojection unit illustrated in FIG. 7A.

[0092]FIG. 8 shows schematically interactive spectacles in accordancewith a seventh embodiment.

[0093]FIG. 9 shows schematically interactive spectacles in accordancewith an eighth embodiment.

[0094]FIG. 10A shows schematically interactive spectacles in accordancewith a ninth embodiment.

[0095]FIG. 10B is a front view of spectacles in accordance with a ninthembodiment.

[0096]FIG. 11A illustrates the naturally perceived field of view of auser of an information system designed in accordance with a tenthembodiment.

[0097]FIG. 11B illustrates the naturally perceived field of view of auser of an information system designed in accordance with a tenthembodiment.

[0098]FIG. 11C is a schematic representation of a scan pattern.

[0099]FIG. 11D is a schematic representation of a modified scan pattern.

[0100]FIG. 12A illustrates the naturally perceived field of view of auser of an information system designed in accordance with an eleventhembodiment.

[0101]FIG. 12B illustrates the naturally perceived field of view of auser of an information system designed in accordance with an eleventhembodiment.

[0102]FIG. 12C illustrates the naturally perceived field of view of auser of an information system designed in accordance with an eleventhembodiment.

[0103]FIG. 12D illustrates the naturally perceived field of view of auser of an information system designed in accordance with an eleventhembodiment.

[0104]FIG. 12E illustrates the naturally perceived field of view of auser of an information system designed in accordance with an eleventhembodiment.

[0105]FIG. 13A is the naturally perceived field of view of a user of aninformation system designed in accordance with a twelfth embodiment.

[0106]FIG. 13B is the naturally perceived field of view of a user of aninformation system designed in accordance with a twelfth embodiment.

[0107]FIG. 14A shows schematically an information system in accordancewith the invention in accordance with a thirteenth embodiment.

[0108]FIG. 14B shows schematically an information system in accordancewith the invention in accordance with a thirteenth embodiment.

[0109]FIG. 15 is an information system in accordance with the inventionin accordance with a fourteenth embodiment.

[0110]FIG. 16 is a schematic representation of an information system inaccordance with the invention in accordance with a fifteenth embodiment.

[0111]FIG. 17 is an optical system in accordance with a sixteenthembodiment.

[0112] In the description of the figures, similar or identical objectsare designated with similar or identically ending reference signs. Manyof the illustrated objects comprise symmetrical or complementarycomponents that are distinguished via a supplementary letter, e.g. “L”for left and “R” for right, after the reference sign. If the statementapplies to each individual component of such a symmetrical orcomplementary grouping, the supplementary letter is left out in somecases for the sake of clarity.

DETAILED DESCRIPTION

[0113]FIG. 1 illustrates schematically an interactive data view andcommand system 100 in a broadest sense of the word, as an informationsystem. The information system 100 is embodied in the form of aninteractive spectacle system 120, i.e. interactive spectacles 120, whichcomprise two optical apparatuses 150. Preferably, the opticalapparatuses 150 are respectively located in an inner side of a left 121Lor right 121R temple of the spectacles 120. Depending on the field ofapplication, other arrangements of the optical apparatuses that do notdisturb the view, e.g. in the region of a bridge 122 of the spectacles120 that crosses the root of the nose of a user are also appropriate.

[0114] The optical apparatus is connected to a processor unit 140 viaconnection lines 101. If the optical apparatuses comprise photodetectorsand/or light sources, the connection lines serve the transmission ofelectrical detection and/or control signals. The photodetectors and/orlight sources can, however, be located in the processor unit 140 and beconnected to the optical apparatuses 150 of the spectacles 120 vialight-conducting connection lines 101. This contributes to reducing theweight of the spectacles 120.

[0115] Still referring to FIG. 1, a communication interface 196(indicated by dash-dots) to which corresponding output signals from thesignal processing unit 140 are transmitted in order to establish acommunicative connection with an external information unit 198 via apreferably mobile communication unit 197 that is connected to theinterface 196 via a signal line 19 a. In the most general case, theexternal information unit 198 is a database or data files that can beaccessed via suitable protocols and/or over the Internet.

[0116] A portable telephone, a laptop or a palmtop can, for example,serve as a mobile communication unit 197, wherein the connection for theremote data transmission can be carried out on the basis of all typicalprotocols such as GSM, UMTS, CDMA, TDMA or DECT. For example, a TDMAcommunication method and apparatus is described in U.S. Pat. No.6,160,800, which is incorporated by reference. A CDMA communicationmethod and apparatus is described in U.S. Pat. No. 5,878,036, which isincorporated by reference. Another mobile telecomunication system isdescribed in PCT Application PCT/SE99/02013 (published as WO 00/28769,which is incorporated by reference. A technique for providing a securelink in a mobile communication system is described in U.S. Pat. No.6,301,479, which is also incorporated by reference.

[0117] The signal line 196 a is bi-directional so that correspondingrequest signals can be returned via the signal line, as an interactivedata exchange between the communication unit 197 and the database 198,to the interface 196 and, from there, to the signal processing unit 140.A control apparatus, not shown in detail and preferably integrated intothe processor unit 140, ensures that the request signals are convertedinto the desired operation signals, with which either the optical unit150 is instigated to place further image information onto the retinaand/or at least a further information reproduction device, such as e.g.a headphone unit 150 a, is instigated to transmit additional informationto the operator. When a further information reproduction system 150 a isprovided, a further signal line 101 preferably exists that is lead tothe processor unit 140.

[0118] A microphone 150 a is indicated by a dash-dotted line that is acomponent of the data view and command system and via which speech inputcan be carried out. A corresponding signal line 101 b is lead from thespectacle system to the processor unit. Speech-controlled operation ofthe interactive data view and command system can be carried out via themicrophone. For example, control operations such as paging throughdisplay menus can be executed or particular system actions such as e.g.the triggering of select or deselect operations can be triggered.

[0119] The information system is capable of processing data from theoptical unit and the communication unit on the fly and simultaneouslyand, as the case may be, as a function of additional control commandsfrom the operator that are either supplied to the processing unit viathe line 101 or the line 101 b, control the signal processing set-up 140via the communication interface ( ) and/or the further informationreproduction unit 150 a such that the desired result of the interactivecommunication is obtainable. It is thus, for instance, possible toconvey acoustic or image information to the operator that has beendownloaded from the Internet via the communication unit upon gaze orverbal command. In the case of optical information, said issuperimposed, precisely positioned, onto image falling onto the eye.

[0120] A further possibility of controlling the data view and signalprocessing set-up—implemented via the components 140, 196, 197—consistse.g. of placing a control menu onto the operator's retina via theoptical unit 150. Since the image on the retina is captured cyclically,preferably with a sampling frequency of 100 Hz, the optical unit 150, inconjunction with the processor unit 140, can determine, at any point intime, which image is situated at the center of the field of vision sincethis image is held in the fovea centralis. When the operator thusfocuses on a particular menu item or a blended-in program control symboland when, at this point in time, a particular trigger signal isgenerated, the corresponding operation is called. The trigger signal canbe generated by a suitable button provided on the system, via the speechinput system (microphone 150 b in connection with a speech decoder inthe processor unit) or else in an optical manner, e.g. by using a blinkof an eye consciously executed at that moment as a trigger.

[0121] Naturally, modifications of the system are conceivable withoutleaving the spirit of the invention. Consequently, the mobile datatransmission set-up 197 can naturally be combined with the interface 196and the signal-processing set-up 140 to a unit, as indicated by thedash-dotted line Z.

[0122] The data transmission set-up can comprise a preferably portablecomputer such as e.g. a laptop or palmtop that is equipped with asuitable remote data transmission interface. An electromechanicalcontrol unit such as e.g. a control mouse can also be provided as acontrol unit for activating particular operations of the system.

[0123] Since the optical apparatus constantly samples the operator'sretina, it can capture the retina's structure—particularly when asuitable wavelength band, for instance the infrared band, is chosen forthe scan beam—so that unique associable user/wearer ID data relating tothe operator can be intermediately stored as a data record in the signalprocessing unit 140. The data record can then be employed for weareridentification/for personalized set-up of the system to the respectiveoperator.

[0124] The system is suited, in a particularly advantageous manner, foruse in the medical field, in particular in the field of ophthalmology,as a therapeutic or analytic device, as well as in military applicationsrelating to accessing large amounts of data in a mobile environment andto using corresponding data downloaded from the database for theoperator as contemporaneously as possible.

[0125] In the following, various embodiments of the optical apparatusare described that can be used in an advantageous manner in combinationwith the data view and command system in accordance with the invention.Furthermore, information systems are described that contain various,equally applicable, modifications of the optical unit and that aresensibly combinable with the data view and command system. For betterunderstanding of the manner of operation of the optical apparatus andthe system combined therewith, however, attention will first be given toFIG. 2 and, based on this representation, the structure of the human eyewill be elucidated.

[0126] To fully understand the information system 100, FIG. 2 provides adetailed view of an eye 280 in cross-section. The eye 280, which issituated in two eye sockets 20 (lat. orbita) formed of skull bone in thehead of a person and which is to be understood here in the sense of aneyeball 280, consists of a chamber surrounded by a translucent cornea283 and a visibly white sclera 28. The sclera 28 is covered on its sidefacing the inside of the eye 280 by a choroid coat 287 that supports,also on its inner side, a light-sensitive retina 281 and supplies samewith blood. Due to its pigmentation, the choroid coat 287 hinders ascattering of the light falling thereon that could disturb visualacuity.

[0127] The tissue of the retina 281 comprises two types of photoreceptorcells, i.e. rods and cones (not shown), which provide the person withthe ability to see. These photoreceptor cells absorb the light focusesby an eye lens 282 in a range of wavelengths from roughly 380-760 nm andconvert it, via a chain of chemical reactions, into electric nervesignals. The signals of the various nerve cells of the retina 281 arethen passed on to the brain via an optic nerve 25 and processed there toa perceptible image. The numerous, highly light-receptive rods, roughly120 million in number, are specialized for signal detection in twilight(so-called scotopic vision) and yield a grey-scale image. The roughly6.5 million, comparatively less light-receptive cones, in comparison,are responsible for color vision during daylight (so-called photopicvision). During light absorption, an oxidation of pigments in thephotoreceptor cells takes place. For the regeneration of the pigments,the cones require roughly six minutes and the rods require roughly 30minutes. An observation period of roughly 200 milliseconds is necessaryuntil the visual stimulus via the photoreceptors sets in and a receptionof information via the retina 281 takes place.

[0128] The retina 281 comprises a depression 286 that appears somewhatmore strongly pigmented due to its higher density of cones in comparisonwith the rest of retina. This depression 286, which is typically calledthe fovea centralis, lies in a region of the retina known as the maculaand represents the region of keenest vision. The fovea centralis 286 isonly occupied by cones, comprises a very high cone density andencompasses solely roughly 0.01% of the retina surface. The optic nerve25 enters into the inside of the eye via a sieve-like opening in thesclera 28 in an area vis-à-vis the lens 282 designated with thereference sign 288. This area 288 does not comprise any photoreceptorcells, whence it is named “blind spot.”

[0129] The chamber formed by the cornea 283 and the sclera 28 ispartitioned by a deformable lens 282 and muscular ciliary processes 23that supports the lens 282. The portion of the chamber lying the lens282 and the retina 281, which makes up roughly two-thirds of theeyeball, forms a so-called vitreous humor 21, a gelatinous structurethat consists, to over 98%, of water and that supports and protects theretina 281. The portion of the chamber lying between the cornea 283 andthe lens 282 carries the name anterior chamber 22 and contains a fluidthat nourishes the cornea 283. In its neutral shape, the lens 282typically refracts the light falling on the eye such the far-away fieldof view is sharply imaged onto the retina 281. Throughcontraction/relaxation of the muscles of the ciliary processes 23, theshape and thus also the refractive characteristics of the lens 282 canbe varied over a wide range in order to allow e.g. a sharp imaging ofclose-lying objects in the field of view onto the retina 281. In mostcases, the person affected is unaware of this process.

[0130] An aperture 285 of variable diameter and consisting of coloredtissue regulates the light falling onto the light-sensitive portions ofthe eye 280 and gives the eye 280 its characteristic coloring. Thusaperture 285 is located in the anterior chamber 22 directly in front ofthe lens 282 and is called the iris 285. Due to the low amount of lightbackscattered by the lens 282, the vitreous humor 21 and the retina 281,the central region of the iris 285 appears black and called the pupil284. The regulation of the pupil size is also carried out by the personsubconsciously.

[0131] The eye 280 is connected to the skull via six muscles 24 that runpartially parallel and partially oblique to one another that allow apivoting of the eye 280 and subsequently a change of the gaze direction.The binocular field of view captured without moving the eyes 280encompasses roughly 170° horizontally and roughly 110° vertically. Ifthe eyes 280 are moved, a binocular field of vision of roughly 290°horizontally and roughly 190° vertically can be captured. The region ofkeenest vision captured by the fovea centralis 286 encompasses solelyroughly 1°. A fictitious axis through the center of this region iscalled the visual axis and corresponds to the direction of gaze. Arotation of the eye around the visual/optical axis is also enabled viathe muscles 24.

[0132] The six muscles 24 are responsible for all movements of the eye.During observation of a fixed point, so-called micro-tremors of the eye280 take place, during which the eyes 280 tremble lightly in order toavoid a temporary exhaustion of the ability of the affectedphotoreceptor cells to chemically react to a persistent stimulus. Duringa change of the direction of gaze or a movement of the head, so-calledsaccadic movements take place, with whose aid the fovea centralis 286 isdirected to its new target of fixation or held on its previous target offixation. During these highly complexly structured movements, the eye280 is involuntarily moved back and forth at a small amplitude of up toseveral tens of degrees and with an extremely fast angular velocity ofup to several hundred degrees per second. During the tracking of amoving object, the eye 280 achieves angular velocities of only one totwo hundred degrees per second.

[0133] To protect the eyeball 280, people have a movable fold of skin,i.e. an upper lid 27 a and a lower lid 27 b that allow a closing of theeye socket 20 against external influences. The lids 27 a and 27 b closeas a reflex in the presence of foreign objects or strong light.Moreover, the lids 27 a and 27 b provide, via regular, typicallyinvoluntary blinking, for an evenly distributed film of tears on thecornea 283 that washes the outer surface of the cornea 283 and protectsit from drying out. The lids 27 a and 27 b also comprise lashes 27 cthat also protect the eye 280 from dust. A conjunctiva 26 covers thespace between the lids 27 a, 27 b, the eye socket 20 and the eyeball280. The conjunctiva 26 merges, on the one hand, with the inner side ofthe lid and, on the other hand, with the cornea 283 and represents asecond wall of protection against the penetration of germs and foreignobjects.

[0134]FIG. 3 shows schematically a first embodiment of the opticalapparatus of the, as described above, interactive spectaclessystem/spectacles 320 as a component of the interactive data view andcommand system. A signal capturing apparatus in the form of a scanningeye scanning apparatus 350D is provided. The left half of FIG. 3represents a plan view onto the head of a user 302 together withspectacles 320 having a right temple 321R, whereas the right half ofFIG. 3 reflects a cross-section of the spectacles 320 running throughthe left temple 321 L. No further components of the information system100 are shown in FIG. 3 other than the apparatuses belonging to theinteractive spectacles 320.

[0135] In accordance with the illustrated embodiment, light beams 333 aand 333 b falling onto the eye 380 that originate e.g. from the field ofview are sharply imaged onto the retina 381 by the lens 382 as acorrelated image and are reflected back by the retina 381 as a retinalreflex image. A light beam 331 that has been reflected back in thismanner passes again, in the opposite direction, through the lens 382, isfocused via two concave mirrors 322 and 323 that belong to the mirrorsystem of the spectacles 320 and is directed, as shown, onto a scanningeye scanning apparatus 350D. The eye scanning apparatus 350D comprises asignal capturing apparatus 351 in the form of a photodetector 351 thatcaptures the light beam 331 reflected back from the retina 381 as wellas two movable flat mirrors 352H and 353V that effect ahorizontal/vertical deflection of the light beam 331 onto thephotodetector 351. In accordance with the embodiment of FIG. 3, thespectacles 320 additionally comprises a light trap 324 that prohibits anincidence of light from undesired directions of incidence. To simplifythe mirror system of the spectacles 320, this mirror 323 can beimplemented via a mirrored inner surface of the spectacle lens. However,the surface must have a particular shape in order to facilitate acapturing of the entire retinal reflex image even when the eye 380 ispossibly in a skewed position. This, on the other hand, limits thedesign freedoms of the spectacles 320.

[0136] A serial, point-by-point scanning of the retinal reflex image asa pixel sequence is carried out via the combination of a point-shapeddetector 351 and corresponding control of the flat mirrors 352H and352V. Preferably, the retina 381 is scanned with a circular, spiral orelliptical scan pattern as described in the PCT ApplicationPCT/EP97/04188 (published as WO98/05992) and U.S. application Ser. No.09/462,440. This has the advantage that the flat mirrors 352 can bedriven without jerky movements and that a higher pixel density (numberof pixels per unit of area of the retina) can be captured in the regionof the fovea centralis 286.

[0137] Preferably, a suitable synchronization operation for determiningthe current optical axis is carried out prior to the capturingoperation—to the respect that it has not already been carried out in aprevious projection operation—so that the scan operation can be carriedout centered to the eye.

[0138]FIG. 4 shows schematically an embodiment of the interactivespectacles 420 in which an output apparatus is in the form of a scanningprojection apparatus 450P. The left half of FIG. 4 represents a planview onto the head of a user 402 together with spectacles 420 having aright temple 421R, whereas the right half of FIG. 4 reflects across-section of the spectacles 420 running through the left temple421L. No further components of the information system 100 are shown inFIG. 4 other than the apparatuses belonging to the interactivespectacles 420.

[0139] In accordance with the illustrated embodiment, a scanningprojection apparatus 450P comprises a light source 453, e.g. a laserdiode or an LED focused via a lens system that emits a projection lightbeam 432 as well as two movable flat mirrors 454H and 454V. Theprojection light beam 432 is directed via the movable flat mirrors 454Hand 454V onto a mirror system of the spectacles 420 that comprises twoconcave mirrors 422 and 423 that projects the projection light beam 432onto the lens 482 of an eye 480 and, in the end, onto the retina 481. Tosimplify the mirror system of the spectacles 420, this mirror 423 can beimplemented via a mirrored inner surface of the spectacle lens. However,the surface must have a particular shape in order to facilitate acapturing of the entire retinal reflex image even when the eye 480 ispossibly in a skewed position. This, on the other hand, limits thedesign freedoms of the spectacles 420. To avoid the incidence of lightthat would be disturbing, the spectacles 420 can be equipped with alight trap 424 that hinders the incidence of from undesired direction ofincidence.

[0140] A serial, point-for-point projection of an image is carried outvia the combination of a point-shaped light source 553 withcorresponding control of the flat mirrors 452H and 452V thatrespectively effect a horizontal/vertical deflection of the projectionlight beam 432. The projection is preferably carried out, as describedin U.S. Pat. No. 6,227,667 and U.S. application Ser. No. 09/462,440, ina circular, spiral or elliptical scan pattern. This has the advantagethat the flat mirrors 452 can be driven without jerky movements and thata higher pixel density can be projected onto the retina 481 in theregion of the fovea centralis 286.

[0141] The degree of perception of an image projected into the eye 480can be controlled in relation to the naturally perceived image via thebrightness of the projected pixels. However, retinal perception is ahighly complex process in which psychological effects also play a strongrole. In this respect, reference is made to the relevant literature ofthe field.

[0142] In simplified form, however, one can say that the retina 481adapts to the brightness of the total light falling thereon. It isknown, for example, that the slight glow of the clock of a radio alarmthat cannot even be perceived in daylight can appear to illuminate anentire room in a dark night. On the other hand, the strong light ofheadlights of approaching vehicles is barely perceptible in daylight.The brightness of a single pixel is thus perceived in relation to thepixels otherwise perceived. The retina 481 functions similarly whenobserved locally. If the brightness of a pixel projected onto a regionof the retina 481 exceeds the brightness of the light otherwise fallingonto this region by roughly 10%, then solely the projected pixel iseffectively perceived by this region of the retina 481 in lieu of theother light. Due to psychological effects, the exact value can also liebetween 5%-10%, 10%-15% or even 15%-20% instead of at 10%.

[0143] Preferably, a suitable synchronization operation for determiningthe current optical axis is carried out prior to the projectionoperation—to the respect that it has not already been carried out in aprevious scanning operation—so that the projection operation can becarried out centered to the eye.

[0144]FIG. 5A shows schematically interactive spectacles 520 inaccordance with a fourth preferred embodiment in which a combined signalcapture and projection apparatus 550 is attached to the spectacles 520in the region of the bridge 522. Referring to FIG. 5B, the combinedsignal capture and projection apparatus 550 comprises both a projectionapparatus 553 as well as a signal capturing apparatus that are housedtogether in a protective housing 558. Light beams 530 make their wayinto the inside of the housing 558 and vice-versa via a translucentwindow 559 in an outer wall of the housing 558. The sealing of thehousing 558 via the window 559, however, keeps dust, sweat and otherforeign materials from disturbing operation of the combined signalcapture and projection apparatus 550.

[0145] Light beams 530, 530 a, 530b are captured/projected analogouslyto the described systems in accordance with FIGS. 3 and 4. Theinteractive spectacles 520 can be simplified, however, in theirconstruction by replacing the mirrors 352/452, which are separate in theprior art, for vertical/horizontal deflection of the respective lightbeams 331/432 with a swiveling mirror 552/554. For the purpose ofachieving a compact design, a partially transmissive mirror 556 canserve to allow separate beam paths within the housing 558 for the light530 falling or projected through the window 559. Preferably, the innerside of the spectacle lens is provided with a surface 523 that stronglyreflects beams incident from this direction that is used as a mirror forthe beam path between the eye 580 and the combined signal capturing andprojection apparatus 550. This contributes to a reduction of thecomponents required and results, in the illustrated embodiment, in asimplified transmission-efficient beam path 530 in which the light beam530 between the eye 580 and the projection/signal capturing apparatus553/551 is only reflected three times. As described above, however, thisresults in a limitation of the design freedoms of the spectacles 520.

[0146] The freedom of movement necessary for a swiveling motion of themirror 552, 554 can be achieved, for example, via a Cardan joint orspring suspension of the mirror 552, 554. Possible embodiments of such aswiveling mirror are known to the person skilled in the art, e.g. fromthe field of microtechnology. Further solutions to the presentdeflection problem in which the respective light beam 530 is deflectedon the basis of electrochrome, holographic, electro-holographic or otherlight refraction or light reflection mechanisms are easily conceivableand equally applicable.

[0147] Although the interactive spectacles 520 are shown in a minimalistembodiment in which a combined signal capturing and projection apparatus550 is solely provided for the left eye 580, it is self-evident that asecond combined signal capturing and projection apparatus 550 having amirror image design can be provided for the non-illustrated right eye,if necessary, in the region of the right half of the bridge 522.

[0148]FIG. 6A shows schematically a modification of the spectacles 520illustrated in FIGS. 5A and 5B. Interactive spectacles 620 in accordancewith a fifth preferred embodiment utilize the left combined signalcapturing and projection apparatus 550L situated in the region lyingbetween the left spectacle lens 624L and the left temple 621L and theright combined signal capturing and projection apparatus 650R situatedin the region lying between the right spectacle lens 624R and the lefttemple 621R.

[0149] Such an arrangement of the combined signal capturing andprojection apparatuses 650L, 650R vis-á-vis the respective spectaclelenses 624L, 624R and the respective eyes 680 is typically associatedwith the necessity of either providing several mirrors along the beampath 630 (cf. mirror 322 and 323 in FIG. 3) or bestowing the respectivespectacle lens 624L, 624R with a particular form in order to guarantee acapture of all regions of the retina 681. This, however, significantlylimits the design freedoms of the spectacles 620. In order to circumventthis problem, the interactive spectacles 620 in accordance with FIG. 6propose spectacle lens 624L, 624R whose inner sides are provided with arespective holographic coating 623L, 623R. Such a holographic coating623 is capable of emulating an arbitrary reflection topology. Forexample, a holographically coated, flat surface can act like aspherically curved surface. Similarly, a holographically coated,spherically curved surface can act like a flat surface. The change ofthe effective reflection topology depends solely on the holographiccontent of the coating. In accordance with the Figure, the holographiccoating 623L and 623R are designed and situated as mirror images to oneanother.

[0150]FIG. 6B shows schematically the combined signal capturing andprojection apparatus 650L. Analogously to the combined signal capturingand projection apparatus 550, illustrated in FIG. 5B, apparatus 650Lcomprises a housing 658, a projection apparatus 653 and a signalcapturing apparatus 651, respective swiveling mirrors 652 and 654, apartially transmissive mirror 656 and a housing window 659.

[0151]FIG. 7A shows schematically a modification of the spectacles 520shown in FIGS. 5A and 5B. Interactive spectacles 720 in accordance witha sixth, preferred embodiment utilize the left combined signal capturingand projection apparatus 750L situated in the region lying between theleft spectacle lens 724L and the left temple 721L and the right combinedsignal capturing and projection apparatus 750R situated in the regionlying between the right spectacle lens 724R and the left temple 721R.

[0152]FIG. 7B shows schematically the combined signal capturing andprojection apparatus 750L. Analogously to the combined signal capturingand projection apparatus 550 illustrated in FIG. 5B, it comprises ahousing 758, a projection apparatus 753 and a signal capturing apparatus751, respective swiveling mirrors 752 and 754, a partially transmissivemirror 756 and a housing window 759.

[0153] The problem of the beam path 730 touched upon above is solved inthis embodiment in a space-saving manner via a special design of pads725L and 725R. Typically, spectacles 720 are supported on the root ofthe nose either through the bridge 722 or through so-called pads 725. Intheir typically commercial design, pads are relatively flat, slightlycurved and oval. Moreover, they are either hingably or swivably mountedon a projection extending from the bridge 722 in order to ensurecomfortable contact of the pads to the side surfaces of the root of thenose. In the illustrated embodiment, the pads 725 are formed asfixed-shaped, elongated units that project from the spectacles 720 inthe direction of eye 780 in the region of the bridge 722. On theirrespective longitudinal sides facing the nose, the pads 725 form thecontact surfaces that support themselves on the root of the nose. Intheir end region lying across from the spectacles 720, the pads 725comprise a support surface on the respective side facing the eye that isprovided with a mirror or a mirroring coating, e.g. a metallic coatingor a holographic coating.

[0154] Although the frame of the spectacles 720, including the pads 725,has a principally fixed form, both quasi-static, e.g. due to materialfatigue and/or temperature changes, as well as dynamic deformations ofthe frame. Particularly when the spectacles 720 are put on and duringactivities in which vibrations are commonplaces, changes to the relativearrangement of the respective spectacle components to one anotherresult. The relative location of the spectacles 720 vis-à-vis the eye780 is also not constant. Accordingly, both the optical system of thespectacles 720, ie. those system components that contribute to theoptical signal capturing and/or the optical projection as well as anyprocessing system connected to thereto, must be conceived and designedsuch that such changes in the arrangement can be taken intoconsideration and/or compensated and/or do not cause any extraordinaryoperational disturbances. This also holds for all types of interactivespectacle systems.

[0155] The problem addressed above can be overcome in particular througha suitable signal processing of the captured signals and the signals tobe generated. Furthermore, an optical marker fixed disposed on thespectacle frame in the vicinity of the typical beam path 730 can beadditionally detected on a regular basis or on demand via the signalcapturing apparatus 751 for the purpose of calibrating its opticalsystem.

[0156]FIG. 8 shows schematically a modification of the spectacles 520illustrated in FIGS. 5A and 5B, interactive spectacles in accordancewith a seventh preferred embodiment in which the signal capturingapparatus 851 of the combined signal capturing and projection apparatus850 is capable of at least partially capturing the corneal reflex image.

[0157] The cornea is typically formed rotationally symmetric to theoptical axis. Beams that fall perpendicularly onto a central region ofthe cornea are thus confocal to the optical system of the eye 880 andform the basis of the image truly perceived on the retina 881. Moreover,the cornea 883 consists, to a large degree, of water and exhibits, forthis reason, a very high degree of reflectivity at a wavelength ofroughly 1.1 μm. Since this wavelength lies in the infrared spectralregion, a capturing of the corneal reflex image is primarily suitablefor infrared applications, e.g. for night vision devices. Reflectionsoccur not only on the outer, concave corneal surface, however, but alsoon the inside of the cornea. Moreover, due to its structure, the cornea883 does not effect mirror-like reflections but instead effects adiffuse reflection that becomes more diffuse with increasing depth ofthe act of reflection within the cornea.

[0158] In order to obtain a meaningful corneal reflex image, effectivelyonly those beams that fall perpendicularly onto a central region of thecornea are captured in the illustrated embodiment. This is achievedthrough several measures. Firstly, the spectacle lens 824 situated infront of the eye whose side facing the eye 880 is provided with asurface 823 that is high reflective for beams incident from thisdirection comprises a specially designed shape that focuses the lightperpendicularly reflected from the cornea such that it falls onto thesignal capturing apparatus 851 as light beams 834 that run nearly inparallel whereas light that is reflected non-perpendicularly from thecornea is deflected in another direction. Alternatively, the spectaclelens 824 can be designed in another fashion, yet comprise a partiallytransmissive, holographically reflecting layer 823 that likewise effectssuch a focusing of the light reflected perpendicularly from the corneasuch that it falls onto the signal capturing apparatus 851 as lightbeams 834 that run nearly in parallel, whereas light non-perpendicularlyreflected by the cornea is deflected in another direction. Secondly, anaperture 857 is provided shortly in front of the signal capturingapparatus 851 that prohibits a capturing of those light beams whoseincident angle lies outside a narrow range of incident angles of thelight beams 834 that run nearly in parallel as described above.

[0159]FIG. 8 shows schematically a modification of the spectacles 520illustrated in FIGS. 5A and 5B. The interactive spectacles in accordancewith another preferred embodiment utilize the signal capturing apparatus851 of the combined signal capturing and projection apparatus 850capable of at least partially capturing the corneal reflex image.

[0160] The cornea is typically formed rotationally symmetric to theoptical axis. Beams that fall perpendicularly onto a central region ofthe cornea are thus confocal to the optical system of the eye 880 andform the basis of the image truly perceived on the retina 881. Moreover,the cornea 883 consists, to a large degree, of water and exhibits, forthis reason, a very high degree of reflectivity at a wavelength ofroughly 1.1 μm. Since this wavelength lies in the infrared spectralregion, a capturing of the corneal reflex image is primarily suitablefor infrared applications, e.g. for night vision devices. Reflectionsoccur not only on the outer, concave corneal surface, however, but alsoon the inside of the cornea. Moreover, due to its structure, the cornea883 does not effect mirror-like reflections but instead effects adiffuse reflection that becomes more diffuse with increasing depth ofthe act of reflection within the cornea.

[0161] In order to obtain a meaningful corneal reflex image, effectivelyonly those beams that fall perpendicularly onto a central region of thecornea are captured in the illustrated embodiment. This is achievedthrough several measures. Firstly, the spectacle lens 824 situated infront of the eye whose side facing the eye 880 is provided with asurface 823 that is high reflective for beams incident from thisdirection comprises a specially designed shape that focuses the lightperpendicularly reflected from the cornea such that it falls onto thesignal capturing apparatus 851 as light beams 834 that run nearly inparallel whereas light that is reflected non-perpendicularly from thecornea is deflected in another direction. Alternatively, the spectaclelens 824 can be designed in another fashion, yet comprise a partiallytransmissive, holographically reflecting layer 823 that likewise effectssuch a focusing of the light reflected perpendicularly from the corneasuch that it falls onto the signal capturing apparatus 851 as lightbeams 834 that run nearly in parallel, whereas light non-perpendicularlyreflected by the cornea is deflected in another direction. Secondly, anaperture 857 is provided shortly in front of the signal capturingapparatus 851 that prohibits a capturing of those light beams whoseincident angle lies outside a narrow range of incident angles of thelight beams 834 that run nearly in parallel as described above.

[0162]FIG. 9 shows schematically a modification of the spectacles 520illustrated in FIGS. 5A and 5B. The interactive spectacles in accordancewith another preferred embodiment utilize a spherical orspherical-acting, partially transmissive, mirroring supplementaryelement 929 arranged between the spectacle lens 924 and the eye 980.Preferably, the supplementary element 929 is arranged confocal to theoptical system of the eye 980.

[0163] The degree of reflectivity of the supplementary element 929 canbe adapted to the requirements of the information system. One can choosebetween a high degree of reflectivity, which allows very good capturingof light beams 933 a-933 c directed onto the eye 980, and a low degreeof reflectivity, which avoids impairment of the perception carried outby the eye 980. Preferably, the supplementary element 929 exhibits a low(e.g. less than 10%), homogenous degree of reflectivity over its entirereflective surface. On the other hand, reflecting organs of the eye 980,for instance the cornea 983 or the retina 981, exhibit, in part, verystrong local reflective dependencies. Similar statements hold for thespectral reflective dependencies of the supplementary element and/or thereflecting organs of the eye 980. Whereas the supplementary element 929can be preferably designed such that it exhibits a homogeneous degree ofreflectivity over all relevant spectral ranges, the various organs ofthe eye 980 exhibit highly differing degrees of absorption that, in manycases, are also subject to strong local variations.

[0164] Excepting partial reflection, the supplementary element 929should have as little effect as possible on the light falling thereon.For this reason, the supplementary element 929 is preferablymanufactured of a homogenous, translucent and uncolored material and ismanufactured to have a constant thickness in the direction of the lightbeams directed toward the center of the eye. By applying ananti-reflective coating to the side of the supplementary element 929facing the eye 980, improved translucency can be achieved.

[0165] The reflecting contour of such a supplementary element 929 iswell defined and can thus be supplied to the information system as knowninformation, whereas the contour of the relevant reflecting organs ofthe eye 980 must first be determined. In some respects, latterencompasses significant difficulties. The capturing of the light beams933 a-933 c directed onto the eye 980 can thus yield valuable images ofthe field of vision.

[0166] In the illustrated embodiment, effectively only those beams thatfall perpendicularly onto the supplementary element 929 are captured.This is achieved through the following measures:

[0167] Due to the partially reflective surface of the supplementaryelement 929, a corresponding portion of those beams 933 a-933 c thatfall perpendicularly onto the surface of the supplementary element 929are reflected back perpendicularly, whereas other beams are reflectedback from the surface of the supplementary element 929 correspondinglyskewed in accordance with the law of reflection “The angle of incidenceequals the angle of reflection.” The light beams reflected backperpendicular to the surface of the supplementary element 929 travelback the same way they came and are thus incident upon the spectaclelens 924 situated in front of the eye. The side of the spectacle lens924 facing the eye 980 is provided with a surface 923 that is highlyreflective for beams incident from this direction and comprises aspecially designed shape or a specially formed coating that focuses thelight beams reflected perpendicularly by the supplementary element suchthat they fall onto the signal capturing apparatus 951 as light beams934 that run nearly in parallel whereas light beams reflectednon-perpendicularly by the supplementary element are deflected inanother direction. In addition, an aperture 957 is provided shortly infront of the signal capturing apparatus 951 that prohibits a capturingof those light beams whose incident angle lies outside a narrow range ofincident angles of the light beams 934 that run nearly in parallel asdescribed above.

[0168] If the image of the field of view captured via the supplementaryelement 929 is to be the basis for a projection correlated with theactually perceived field of view, then the correlation between thecaptured light and the perceived field of view must be determined. Inaccordance with the embodiment of FIG. 9, this correlation is achievedthrough a preferably confocal arrangement of the supplementary element929 to the optical system of the eye 980. Thus, preferably, thesupplementary element 929 is fastened to the spectacles via anadjustable suspension such that the position of the supplementaryelement 929 can be adjusted in both vertical as well as in twohorizontal directions.

[0169] To obtain confocal arrangement, the supplementary element 929 issituated rotationally symmetric to the optical axis and is spaced fromthe lens 982 such that the optical mid-point of the optical system ofthe eye agrees with the mid-point of the sphere defined by the sphericalor spherical-acting supplementary element. The optical axis can besufficiently determined for this purpose via the orientation of thepupil 984 that is easily recognizable via its sharp contours and whoseorientation is easily determinable due to its round shape. In addition,due to the spherical or spherical-acting shape of the supplementaryelement 929, no pivoting of the supplementary element 929 around thepossible pivotal axes of the eye 980 is necessary to ensure confocalitysince, even the case of a skewing of the eye, at least a substantialportion of the supplementary element 929 remains, in terms of optics,rotationally symmetric to the optical axis through a correspondingvertical and/or horizontal shift of the supplementary element 929. Asregards the distance to the lens 982, there are various possibilitiesfor determining the necessary distance. For example, an optical oracoustic measurement of the cornea 983 can be carried out whosecurvature yields a very good estimate of the correct location of thesupplementary element 929. Retinal or cornea reflex images can also beat least partially captured, and the correct distance can be determinedon the basis of a comparison of the reflex images with the lightcaptured via the supplementary element 929.

[0170] Due to the spherical or spherical-acting implementation (e.g.through a holographic coating) of the partially reflecting surface ofthe supplementary element 929 as well as through the confocalarrangement of the supplementary element to the eye 980, solely thosebeams 933 a-933 c that fall perpendicularly onto the surface of thesupplementary element 929 are confocal to the optical system of the eye980 and thus correspond to the beams falling onto the retina.

[0171]FIG. 10A is a plan view and FIG. 10B is a front view of spectacles1020 in accordance with another embodiment utilizing two sensor devices1061R and 1061L. For example, two solid-state cameras (e.g., CCD or TTLcameras) are provided to capture additional signal, in particular in thevisible field of vision. FIG. 10B also shows the left eye 1080L and theright eye of a wearer 1002 wearing the spectacles 1020. For the sake ofclarity, however, no other features of the user 1002 are represented inFIG. 10B.

[0172] To avoid the occurrence of parallax between the images capturedby the respective cameras 1061R, 1061L and the images received by theeye associated therewith, the cameras 1061 should be arranged ascoaxially as possible to the eyes with regard to their “optical axes.”In view of the system size of such solid-state cameras 1061 and thecurrent state of the art, it has turned out to be meaningful to locatedthe cameras 1061 in the front region of the respective temples 1021L,1021R as shown. A mounting in the region of the bridge 1022, e.g. in thepads 1025, is also meaningful. After a further miniaturization, thesolid-state cameras 1061 will be able to be located in the spectacleframe over the respective spectacle lenses 1024L, 1024R in order toachieve further axial identity. It is foreseeable that solid-state andother types of light capturing systems will, in the future, be able tobe built into the spectacle lens 1024, which can naturally be glass,plastic or other translucent material. Such an arrangement of thecameras 1061 would allow a signal capturing that is coaxial and nearlyconfocal with the eye 1080L, 1080R.

[0173] In a non-coaxial arrangement of the sensor apparatus 1061 to therespective eyes 1080L, 1080R, the information obtained from the sensorapparatuses 1061 should be brought into correlation with the eyes 1080,as need be. Such correlation is particularly important when the sensorapparatuses 1061 are implemented by cameras 1061 and a superimposedimage based on image information obtained from the cameras 1061 is to beprojected into the respective eye 1080L, 1080R.

[0174] If the image captured by the cameras 1061 is simply projected onthe respective eye 1080L, 1080R, so-called parallax occurs, in which the“field of view” of the respective camera 1061L, 1061R does not agreewith the naturally perceived field of view. In particular during askewing of the eye 1080 deviating from the neutral position, or in thecase of objects lying closer in the field of view, parallax would leadto abnormal perception in the case of superimposition since, in suchcases, the optical axis of the eye 1080 would lie skewed to the “opticalaxis” of the respective camera 1061L, 1061R.

[0175] During correlation in this sense, only the portion of the imagecaptured by the cameras 1061 is projected into the respective eye 1080L,1080R that lies in corresponding “correlation” to the optical axis ofthe respective eye 1080L, 1080R. In the simplest case, an at leastpartial reflex image of the field of view is captured from therespective 1080L, 1080R via the signal capturing apparatus 1051.Characteristic pixels that can be found in both the captured refleximage as well as in the images captured by the cameras 1061 then serveas reference points for a perspectively correct projection of the imageinformation captured by the cameras 1061 onto the eye 1080. Similarly,the signals captured from the eye 1080 can serve to determine the gazedirection of the respective eye 1080L, 1080R with respect to thecoordinate system of the spectacles 1020 in order to carry out amathematically based correlation from this angular information.

[0176] The correlation is also meaningful in the context of systemapplications in which the eyes 1080 are hindered from perceiving thefield of view. This is the case, for example, during use of occluded,so-called “virtual reality” glasses 1020 (as shown, yet withnon-translucent lenses 1024) wherein solely a synthetically generatedimage is presented to the eyes 1080. In such a case, the aforementionedcorrelation can consist, for example, of capturing the gaze direction ofthe eye 1080 as described above and projecting a virtually generatedimage that corresponds to the orientation of the respective eye 1080L,1080R. In this case, however, the spectacles 1020 serve as a coordinatesystem. If, however, the position and orientation of the spectacles 1020is also determined, e.g. on the basis of the images captured by thecameras 1061, then a correlation between the respective eye 1080L, 1080Rand the surroundings can be created. Such a system could be used, forexample, in a virtual amusement house, similar to a house of horrors.Someone who's currently standing on a conveyor belt could have, forexample, a virtual image projected into the eyes that gives him thefeeling he is running on floating tree trunks in the middle of a wildriver.

[0177] We emphasize that the information system described above inconnection with FIGS. 5 to 10 must not necessarily operate with acombined signal capturing and projection apparatus. According to anotherpreferred embodiment, the information system may include separate signalcapturing and projection apparatuses. According to another preferredembodiment, the information system may include only a signal capturingapparatus, and according to yet another preferred embodiment, theinformation system may include only a signal projection apparatus, or inanother embodiment none of the two apparatuses. Any of the signalcapturing or projection apparatuses may execute only partial or limitedcapture or projection.

[0178]FIGS. 11A and 11B illustrate schematically the use of informationsystem 100 a telescope for a user. FIG. 11A shows the naturallyperceived field of view 1190 of a user. Although the field of view 1190encompasses roughly 170° of the surroundings horizontally and roughly110° of the surroundings vertically, solely a small region 1191 ofseveral degrees around the visual axis forms the region of keenest sight1191.

[0179] Via its capturing of light from the field of view and theaforementioned possibility of a projection of image information into theeye, the information system can be designed such that this region 1191is projected, optically enlarged, onto the region of keenest sight 1191after corresponding processing of the captured pixels via an evaluationapparatus comprised by the information unit, e.g. upon pressing of abutton. As described above, the degree of perception of an imageprojected in this manner in relation to the naturally perceived imagecan be controlled via the brightness of the projected pixels. If thefield-of-view light is captured, for instance, as a reflex image fromthe eye, a spatial or temporal separation of the capturing and theprojection will ensure that the projection does not influence thecapturing.

[0180] In the case of a conventional telescope, the spatial relationshipto the surroundings is lost on account of the fact that the entire fieldof view is shown in enlargement. As a consequence, a person lookingthrough a telescope cannot walk or drive at the same time. Thisphenomena is well known.

[0181] Since the information system 100 can determine, by capturingsignals from the eye, the visual axis/the position of the foveacentralis relative to the optical system of the spectacles, theinformation system is capable of avoiding this disadvantage of aconventional telescope. For example, the projection can be carried outin a manner shown in FIG. 11B. The system projects a small region 1191lying directly around the visual axis in the natural field of view ontothe fovea centralis in enlargement, whereas no projected imageinformation is superimposed on the remainder of the field of view. Thescene peripherally perceived by the user thus stays the same in spite oftelescopic presentation of the most relevant region of the field ofview. In order to achieve this effect, the brightness of the imageinformation projected into the eye must naturally be chosen such thatthe desired relationship of perception between the natural and theprojected image results. This system also has the advantage that theamount of image processing necessary for the enlargement is held withinlimits since only a selected image region 1191 of the field of view 1190is processed.

[0182] In accordance with another embodiment, an enlarged image isprojected into the eye such that the projected image in an annularborder region between the region of keenest sight 1191 and the remainingregion of the retina is enlarged more strongly as it gets closer to thevisual axis. In this case, no enlargement takes place along the outeredge and along the inner edge an enlargement takes place with the same“zoom factor” as the enlarged image projected into the inside of thering, i.e. onto the fovea centralis. Thus, when the brightness of theprojected image information is correspondingly chosen, a soft transitionbetween the peripheral scene and that which is telescopically seenresults.

[0183]FIGS. 11C and 11D schematically illustrate the enlargement of theimage naturally falling onto the fovea centralis by modifying a scanpattern 1138, 1139 during the scanning of a reflex image. In FIGS. 11Cand 11D, the projection pattern 1137 and scan patterns 1138, 1139 areillustrated in the same plane, for the sake of explanation. In general,in the information system 100, the projection can take place onto theretina, whereas the scanning can takes place, for example, from thecornea, and this may be done by different units (or even on differenteyes of the wearer).

[0184]FIG. 11C schematically illustrates a typical scan pattern 1138that scans the region 1189 of the cornea or retina reflecting the fieldof view. In this vastly simplified example, it is assumed, for the sakeof comprehensibility, that the respective pixels of the sequentiallyscanned image are projected back, after image-processing preparation ifneed be, in their proper sequence as corresponding images of thesequential image projected into the eye. In the illustrated example, thescan pattern 1138 thus corresponds to the projection pattern 1137 inspite of possible spatial or temporal separation of the scan beam andthe projection beam. If an enlargement of a central region of the fieldof view is desired, then the scanning can be effected in accordance witha modified scan pattern 1139 that effects an increase in the density ofthe captured pixels in that central region. If these pixels captured athigher density are projected back correspondingly, yet at lower densityduring the projection, then an enlarged image is the result.

[0185] In accordance with another embodiment, information system 100 isconstructed and arranged as a guidance system. For this purpose, theinformation unit of information system 100 comprises position sensore.g. acceleration measurement apparatuses or GPS receivers as well as adata bank or data bank connection that supplies orientation data. Such adata bank can be implemented e.g. via a CD-ROM carrying the data, a DVDor another exchangeable storage medium in connection with acorresponding reading device. Methods and apparatuses for obtainingposition fixing information that, for example, determine the currentlocation or allow their determination via a combination of suchorientation data with data obtained from the position sensors are known.In a typical apparatus, the orientation data comprise map informationthat are used for position determination in conjunction with signalssupplied by the position sensors. The establishment of a correlation ora dependency, e.g. when such position information is obtained orpresented, between the signals captured from the eye or light capturedfrom the field of view and the provision of information, however, farexceeds that known to the art.

[0186]FIGS. 12A to 12E show the field of view 1290 perceived by a userof an information system designed in form of a guidance system,according to another embodiment. The system evaluates the capturedfield-of-view light with respect to the positioning information obtainedvia a pattern recognition that takes the data available for thedetermined whereabouts into consideration. Orientation hints such ascharacteristic buildings, side streets, or the like that are to beexpected for the determined whereabouts are thereby recognized such thate.g. visual or acoustic guidance or identification can be carried out,if necessary. These operations may be performed by the information unit.

[0187] In the illustrated example in accordance with FIG. 12A, theguidance system serves for navigation. In this case, it is determined,e.g. based on a computed or predetermined route, available mapinformation and the current whereabouts, that one should turn into thesecond street on the right side. This street is recognized on the basisof the captured field-of-view light via pattern recognition, in responseto which a hinting arrow pointing at the street is locationallycorrectly blended into the field of view via projection taking intoconsideration the gaze direction determined by the system. Similarly,the guidance system could provide the driver with an acoustic message,e.g. “turn right after 50 meters” or “now turn right.”

[0188] In the example illustrated in FIGS. 12B and 12C, the guidancesystem serves to provide information. For example, a user canselectively be provided information about his direct surroundings.Referring to FIG. 12B, a tourist using the information system looks at acharacteristic building and actuates an activation button that isphysically present or virtually blended into the field of view. Thebuilding is subsequently identified on the basis of the determinedwhereabouts and a pattern recognition based on the capturedfield-of-view light or an electronic compass that determines thedirection of the head. In response, information about the building isprovided. This information can originate from a data bank or otherinformation source and can be selected, e.g. interactively, viacontext-dependent menu that visually or acoustically lists theinformation available for selection for that particular building. Theselection can be carried out via voice control or via fixation with theeyes. Further information re eye-controlled menu selection will bedescribed in a later section of this description.

[0189] Referring to FIG. 12B, historic data are blended into the fieldof view via projection. In doing so, the system determines, from thecaptured field-of-view light, a suitable blend-in position, e.g. infront of a monotonous roof or in front of the sky. The data are blendedin in accordance with the blend-in position. Typically, the foveacentralis is not directed at the blend-in position at first, whence theblended-in data is first perceived as an unfocussed, peripheralappearance. The locationally fixed, blended-in data are not imaged uponthe fovea centralis until a corresponding pivoting of the gaze directionin accordance with FIG. 12C. If the gaze is directed at another buildingrecognized by the system, then the blend-in information can change inaccordance with FIGS. 12D and 12E. In the Figures, the circle 1290represents the perceived field of view, whereas the circle 1291designates the region of the field of view captured by the foveacentralis.

[0190] By using a compact and portable design shown in FIG. 1, theinformation system 100 can be used as an orientation system worn by apedestrian, a bicyclist, a motorcyclist or other vehicle driver.

[0191]FIGS. 13A and 13B illustrate the use of the information system 100as a driving aid, in accordance with another embodiment. Preferably, theinformation system 100 includes an information unit comprising adistance sensor (e.g. an optical or acoustic distance measuring device)or a radar apparatus. Alternatively, the information unit is connectedto an external distance measuring system. The distance sensor determinesthe distance between a vehicle and objects located in front of thevehicle in the direction of motion. In the case of a stereoscopiccapturing of light from the field of view, the distance could bedetermined via a computation of parallax in which change of position ofthe object in a respectively captured left and right image conveysinformation re the distance.

[0192] The information unit may also include an evaluation apparatusthat can determine a (or calculate a probability), that the vehicle ison a collision course with an object within the field of view. A user ofthe system has a perceived field of view 1390 (shown in FIGS. 13A and13B). Upon determining a potential collision course, the driving aiddisplays, e.g., a warning symbol 1395, which can be blended in theregion of keenest sight 1391 and a warning circle 1394 can be blended inaround the dangerous object by projection, as described above. If theobject is located outside or on the edge of the region of peripheralvision, then a further warning symbol 1395 a can attention to where thedanger lurks, as shown in FIG. 13A.

[0193] Other information relevant to driving safety can also bedetermined via sensors or the captured field-of-view light. For example,an evaluation apparatus could recognize the road lane markings of a roadlane lying within the field of view via pattern recognition and computethe highest allowable speed, in particular in curves, therefrom. If theinformation system determines, independently or via connection to theinstrument system of a vehicle, that the vehicle has exceeded thiscomputed highest speed, then a warning symbol 1395 can be blended in inthe region of keenest vision 1391. This is illustrated in FIG. 13B. Theadvantage of blending in of a warning symbol 1395 in the region ofkeenest vision 1391 lies in the fact that the symbol 1395 appears wherethe eye is looking and thus does not tempt the eye to look away from thepresent scene. For this reason, the brightness of blended in symbolsshould be chosen such that the symbol appears translucent. Optionally,the driving aid system can also acoustically warn a user about thedanger.

[0194] The information system 100 may be a complex, multifacetedinformation system. FIGS. 14A and 14B illustrate the use of theinformation system 100 as a mobile fire department command center,according to another embodiment. The mobile fire department commandcenter 1410 includes a command console 1412 and several helmet systems1411. Each of the helmet systems 1411 comprises a signal capturingapparatus, as described above, and a field-of-view capturing apparatus.Each of the helmet systems 1411 can optionally be equipped with aprojection apparatus, infrared sensors or position sensors. The helmetsystems 1411 can also be equipped with further sensors that allow, e.g.,an assessment of the air quality or wind speed. For the purpose ofcommunication, each of the helmets 1411 is equipped, for example, with aradio transmission unit that communicates with the command center 1410and/or the command console 1412. The radio transmission unit can takeover both tasks of an information unit as well as tasks of an outputapparatus by transmitting and receiving information.

[0195] Preferably, the field-of-view images captured by the respectivehelmets 1411 which can be brought into agreement with the trulyperceived field of view of the respective firemen on the basis of thesignals captured from the eyes are transferred to the command console1412 and presented there on monitors. In order to reduce the amount ofdata to be transmitted, users of the command console 1412 can also weara projecting spectacle system so that solely the image data falling ontothe region of the user's fovea centralis must be captured and/ortransmitted in high resolution. A correlated field-of-view image of anindividual fireman or a mosaic of several images could be projected intothe user's eye. Thus, the user could see exactly that which the firemansees or be provided an image from the fireman's field of vision thatchanges depending on his own eye movements.

[0196] In the optional case of a projection, additional informationcould be woven into the image projected to the user and/or the fireman.For example, orientation and/or temperature information obtained via theposition sensors and/or infrared sensors could be blended into the fieldof view. Constantly blending in particular points on a compass such asNorth and West as well as altitude information could be helpfulreference information both to the user far away from the action he isseeing as well as to the fireman veiled in smoke and haze.

[0197] Through appropriate pre-processing of the captured positioninformation and due to the inherent networking of the system components,the position of his colleagues, e.g. via a characterizing “X,” or theposition and severity of the sighted or otherwise captured hearts of thefire, e.g. via a dot that is colored in accordance with the strength ofthe fire, could be blended in to each fireman. This would make fightingthe fire easier and would reduce the probability of accidentallyinjuring a colleague hidden behind smoke or a wall.

[0198]FIG. 15 shows schematically an information system adapted for theoperation of an external system, e.g. a remote controlled robot 1579designed to move dangerous objects. A movable robot 1570 includes acamera apparatus 1571 as well as a grasp arm 1572. The robot 1570 isconnected to a spectacle system 1520 worn by a user 1502 e.g. via aradio connection. The images captured mono- or stereoscopically via thecamera apparatus 1571 can be mono/stereoscopically projected onto theretina of the user 1502 via a projection apparatus comprised by thespectacle system 1520. In the case of a stereoscopic projection, spatialvision would be ensured.

[0199] If the camera apparatus 1571 is provided with a macroscopic lenshaving a wider “field of view” than the field of view of the user 1502,then the field of view seen by the user 1502 can be held in correlationwith the remote image as a function of the captured eye movements of theuser 1502 via a corresponding selection of an image detail from theimage supplied by the camera apparatus 1571 as described above. Themovements of the head of the user 1502 can also be captured via positionsensors such that the camera apparatus 1571 pivots in correlation withthe head movements. The information system in accordance with theinvention thus offers a previously unachieved degree of visualauthenticity during the perception of a remote scene, which considerablyeases the control of such an external system 1570.

[0200] By attaching a microphone to the external system, in particular adirectional microphone that is directed as a function of the headposition or the gaze direction, in connection with the headphonearrangement on the spectacle system allows a further sensory dimensionto be realized.

[0201] In order to allow further operational control of the robot 1570,a manually operable joystick 1525 is connected to the spectacle system1520 e.g. via a cable 1526. Thus would allow, for instance, the grasparm 1572 or the direction of motion of the robot 1570 to be controlledin several directions.

[0202]FIG. 16 schematically illustrates another embodiment of theinformation system using a spectacle system 1620, which acts as auniversal remote control for one or more devices, for instance acomputer, a video recorder 1676, a printer 1677, a slide projectorand/or a telephone 1679. The spectacle system 1620 provides an interfacethat communicates in two directions between a user 1602 and any device1675-1679 to be controlled. First, the device 1675 through 1679 must berecognized. This is fundamentally carried out, by gazing at the device(1675-1679) to be operated with the fovea centralis. The identity of thegazed-at device (1675-1679) can be determined either with or without theassistance of the device (1675-1679). In the following, it is assumedthat both the device (1675-1679) as well as the spectacles 1620 areequipped with the signal reception and/or transmission apparatusnecessary for the operations described.

[0203] If the identity is determined with the aid of the device1675-1679, then this device 1675-1679 either radiates an ID-signal e.g.an infrared or ultrasonic signal, in more or less regular intervals orit is requested by a request signal radiated by the spectacles 1620 toradiate an ID-signal. The request signal must be radiated localized tothe gaze direction in order to avoid addressing other devices. TheID-signal radiated by the device 1675-1679 is recognized by thespectacles, as a result of which conclusions are made re the identity ofthe device.

[0204] If the identity is determined without the aid of the device1675-1697, then the spectacles 1620 carry out a pattern recognition ofthe gazed-at region of the field of view in cooperation with a databankor other information source 1640 that contains pattern recognition datafor the respectively addressable devices 1675-1679.

[0205] Based on the identity of the device 1675-1679, a menu that isadapted to the possible functions of the device is blended in, at afixed location, into the field of view of the user 1602, if necessaryupon the pressing of a button or the blinking of an eye. If thefunctionality of the spectacles is not readily known, then thecorresponding information is first established from a databank or otherinformation source 1640, e.g. via standardized interrogation of thedevice itself. In this case, identification of the device embedded intothe interrogation signal ensures that solely the desired device respondsto the interrogation. By blending in the menu into the field of view ata fixed location, the user 1602 can control the menu, which may behierarchical if necessary, via slight eye movements like a computermenu.

[0206] After the desired function has been selected, a signalcorresponding to the function is sent from the spectacles 1620 to thedevice 1675-1679. In this case, identification of the device embeddedinto the signal can ensure that solely the desired device reacts to thesignal. In this manner, quick and easy operation of many devices can beachieved with little hardware.

[0207]FIG. 17 shows schematically an optical device with a hinged mirror1755 allows a switching between a capturing from the field of view and acapturing from the eye 1780 or a projection onto the retina 1781. Theadvantage of this optical device lies in that the same swiveling mirrors1754H and 1754V can be used for a capturing from the field of view andfor a projection onto the retina 1781 and that the beam path for acapturing from the field of view and the beam path for a capturing fromthe eye 1780/a projection onto the retina 1781 is, to a large degree,accordingly identical. In this manner, a high correlation between thelight captured from the field of view and the signals captured from theeye/a high correlation between the light captured from the field of viewand the image projected onto the retina is achieved through the opticalsystem itself. This means that no additional correlation errors arecaused by the aforementioned beam paths traveling across differentswiveling mirrors that could exhibit different rotation characteristics.For capturing light from the field of view and capturing light from theeye, even the same light capturing apparatus 1751 can be used. Thecorrelation can solely be negatively influenced by the reflection on thespectacle lens 1724 and the optical system of the eye 1780.

[0208] There are other embodiments directed to the use instead of atraditional TV devices, newspapers, books, as helmets, diagnostic ortreatment devices and other applications.

[0209] Previous electronic books and/or newspapers have the disadvantageof being too heavy and/or too unwieldy and can moreover only present alimited amount of information per page. Portable video and TV devicesare also heavy and/or unwieldy. If the information system in accordancewith the invention is designed such that the provision of informationcomprises a projection of image information into the eye, then variousvision-related media, e.g. electronic books or newspapers, television orvideo games, can be implemented via the information system. In suchcase, the information system in accordance with the invention isimplemented, for example, in the form of wearable spectacles asdescribed above that can be connected e.g. to an information network, aportable storage apparatus, e.g. a CD-ROM or DVD-reading device oranother information source via a cable, infrared or radio connection.

[0210] An advantage of such a design of the information system inaccordance with the invention is that its capturing of signals from theeye in conjunction with a field-of-view capturing allows a projection inwhich the projected text and/or the projected images appear to be fixedin space. For this purpose, the information apparatus comprises anevaluation apparatus that determines the correlation of the visual axisto the field of vision and that accordingly controls the projection suchthat the information projected onto the eye appears to be immovablevis-à-vis the field of vision in spite of movements of the eye. Thedetermining of the correlation of the visual axis to the surroundingscan also be assisted by position sensors mounted in the spectacles.

[0211] The virtual position of the fixation fixed e.g. via a fixationwith the eye in conjunction with a blinking of the eyes or a pressing ofa button or even automatically, for example, by using animage-processing evaluation of the field of vision that determines anarea of the field of vision having as little content as possible. Thedisruptive effect of the natural field of view not necessarily coveredup by the projection of information can be reduced via acomplementary-color “wiping out” in which complementary-colored pixelsare determined on the basis of the light captured from the field of viewwhose correlated projection onto the respectively associated regions ofthe retina make the natural background appear white on account of theaddition of colors. If a black background is desired, then the perceivedtotal brightness of the projection must exceed the perceived totalbrightness of the natural field of view by roughly 10% to 20% asdescribed above so that even the brightest points of the natural fieldof view are perceived as black.

[0212] For the sake of controlling operation, image informationrepresenting virtual control knobs can be projected into the eye suchthat they likewise appear fixed in the vicinity of the text and/or imagein the field of view. The virtual information medium could thus beremote controlled, i.e. page turning, fast forwarding, rewinding or thelike, by gazing at the corresponding control knob with the foveacentralis plus pressing a button or blinking an eye. Similarly, accessto lexical, databanks, etc. could be made possible by gazing atpresented words or image sections. Instead of control knobs, theinformation system could also be controlled, for example, via menuguidance in which control menus “pop-up” when a particular region of theimage is observed in order to allow an ocularly controlled selectionfrom the menu which may be hierarchically constructed, if necessary.

[0213] A further advantage of such a design of the information system inaccordance with the invention is that the amount of data necessary for asufficient, momentary presentation is far less than the amount of datathat would be necessary for a high-resolution presentation of the entirefield of view. This is due to the fact that the information system hasknowledge of the region of keenest sight. Thus, only those portions ofthe projection must be carried out at high resolution that regard theregion of the fovea centralis. Onto the other regions of the retina, aprojection having a lower pixel density suffices. The amount of datanecessary for an instantaneous presentation is accordingly reduced,which has clear system advantages. In particular, the perceived size ofthe projected image may be arbitrarily chosen without unprocessablylarge amounts of data for presentation of the instantaneous image beingthe result.

[0214] If the projected image is larger than the field of view, then thecurrent visual axis determines the cropping of the image. The projectionis carried out such that the current image detail fills the entireactive region of the retina. By moving the eyes, further sections of theimage can be brought into the field of view. If the projected image issmaller than the field of view, then projection must only be carried outonto a limited portion of the retina. If the natural background of thefield of view is not blended out, then this changes during movements ofthe eyes. In particular for television or cinema-like presentations ofinformation, a projection that fills the field of view exactly ispreferred.

[0215] If signals are captured from both eyes of a user, then theprojection can be carried out stereoscopically, wherein a slightlydifferent image is supplied to each eye such that the brain believes toperceives a three dimensional total image. This allows an optimalsystem-human interface e.g. for 3D television, 3D video games, 3D CADapplications or other, in particular interactive, 3D applications to berealized. Preferably, the information system comprises further controlelements, for example a joy stick, pedal or steering wheel that allows anavigation and/or change of perspective within the presented virtualimage or other influencing of the presentation of information or of asystem connected with the information system. As described above, theeye itself can also act as a control element.

[0216] By accordingly applying the above measures necessary for thepositioning of an electronic newspaper at a virtual location, it islikewise possible to project the person wearing the information systemin accordance with the invention other orientation aids onto the retinasuch as, for example, an artificial horizon.

[0217] The information system 100 may be arranged for ophthalmologicalapplications and visual aids. Due to its capturing of signals reflectedback from the eye, the information system in accordance with theinvention is excellently suited for embodiment as an ophthalmologicalsystem. For example, the information system in accordance with theinvention can be implemented as a positioning system forophthalmological surgery, in particular for ophthalmological lasersurgery. The information system can also be used e.g. as anophthalmological diagnostic system, visual aid system and/or visualdeficiency correction system.

[0218] Most of the structures or organs of the eye are very small incomparison to manual movements. Diseases and injuries to thesestructures/organs often only affect a small, microscopic area. Asopposed to many other parts of the body, the eyes, however, cannot befixed, which makes the treatment of possible diseases or injuries to theeye particularly difficult.

[0219] Due to the ability of the information system to exact followmovements of the eye and provide information with regard to themomentary position of the eye even to other systems, these difficultiescan be overcome via a therapeutic system on the basis of the informationsystem. For example, the therapy system can be connected to theinformation system for the purpose of exchanging information in such amanner that the information with regard to the momentary position of theeye is provided to the therapy system such that a high-precision,automated therapy of the eye can be carried out even when the eye ismoving.

[0220] In accordance with another embodiment, a therapeutic laser beamis directed via the optical system. A laser treatment of the eye, inparticular of the retina, can thus be carried out in the same manner asa projection as described above. For example, diseased veins in thechoroid coat can be stultified in that a photosensitive preparation isinjected or taken in and that the diseased portions of the choroid coatare precisely irradiated for several tens of seconds. Such a therapy canbe precisely carried out with the aid of the information system.

[0221] In order to be used as a visual aid and/or visual deficiencycorrection system, the output apparatus of the information systemcomprises a projection apparatus that projects the vision-improvingimage information onto the retina. In addition, the informationapparatus comprises an evaluation apparatus that determines thevision-improving image information on the basis of the light capturedfrom the field of view. The vision-improving image information ispreferably projected onto the retina in correlation with the signalscaptured from the eye such that the naturally perceived field of viewand the projected image information are perceived as a unitary image. Inextreme cases, the vision improving image information is projected ontothe retina such that the otherwise naturally perceived field of view isnot at all perceived by the eye. As described above, the degree ofperception of an image projected in this manner in relation to thenaturally perceived image can be controlled via the brightness of theprojected pixels.

[0222] Such an information system allows e.g. vision deficiencycorrection for short or far-sightedness as well as for color blindnessto be carried out. During the correction of short or farsightedness, theinformation system can set to a (quasi-)fixed correction, can allow avariable correction, or can automatically, dynamically adjust itself tothe visual deficiency. The correction is carried out via an adjustable(if need be), optical focusing system within the projection apparatus orvia image processing measures. Latter can be implemented at low systemcost.

[0223] Implementations with (quasi-)fixed or variable correction areunderstandable to the person skilled in the art without furtherexplanation due to inherent similarity to similar optical systems. Animplementation with a dynamic, automatic correction of the naturalimaging error comprises, in addition to the aforementioned correlation,a further dependency on the signals captured from the eye. In such case,in particular a retinal reflex image is captured that supplies, viacomparison with light captured from the field of view and/or via imageprocessing evaluation, information re the sharpness of the image imagedonto the retina. The light captured from the field of view isaccordingly processed into vision improving image information andprojected onto the retina. The information system can also act as adiagnostic system through output of the correction values determined inthis manner.

[0224] Due to its capturing of signals reflected back from the eye andlight originating from the field of view, the information system inaccordance with the invention is in a position, by using acorrespondingly programmed evaluation apparatus, to supply informationabout many ophthalmologically relevant characteristics of the eye. Forexample, squint angle, primary positions (PP), visual field testing evenwith colors, threshold tests, standardized testing methods for glaucomadiagnosis, retinal function tests (e.g. ERG and VEP) even at selectedlocations and tests of the receptive fields can be carriedout/determined. The person skilled in the art selects the signals to becaptured from the eye for this purpose, the field-of-view stimulinecessary for this purpose and the processing algorithms necessary forthis purpose on the basis of his specialized knowledge, accordinglytaking into consideration the invention described above.

[0225] Whereas e.g. the keenness of the vision can be determined throughan evaluation of signals reflected back from the eye and can besubsequently corrected, the correction of many other visual deficienciespresumes a system-independent determination of the deficiency, forexample by an ophthalmologist. A befitting setting of the correctioncarried out by the information system can be carried recursively orsimply.

[0226] In a recursive adjustment process, a correction is carried out bythe information system in accordance with a previous setting while thevisual acuity of the person with defective vision is being tested. Onthe basis of the results of the tests, a new setting of the informationsystem is chosen. This process is repeatedly carried out until thevisual deficiency has been sufficiently compensated. In this manner, theinformation system acts equally as a diagnostic system since the visualdeficiency can be determined based on the best-correcting final setting.

[0227] In a simple setting processing, the visual acuity of the personwith defective vision is tested without any type of compensation. Basedon the results of the tests, a suitable setting of the informationsystem is chosen that then, in later operation, prepares the lightcaptured from the field of view into vision-improving image informationin accordance with this setting and projects it onto the retina. Duringthe preparation, e.g. particular spectral components or particularregions of the field of view are emphasized or modified through otherimage processing measures in accordance with the setting, i.e. theoriginal visual deficiency.

[0228] For people suffering from night blindness, a visual aid can berealized via the information system in accordance with the invention,for example, in that the light captured from the field of view, e.g. viahighly light-sensitive photo-detectors, is strongly amplified andprojected onto the retina. In this manner, the cones can be stimulatedin such manner that predominantly color, photopic vision instead ofscotopic vision takes place. Also, the maximally allowable brightness ofthe individually projected pixels is limited to a predeterminedthreshold value in order to avoid a glaring through brightlyilluminating objects such as street lamps and oncoming cars. Such asystem is thus also suitable as an anti-glare system since, if thebrightness of the entire field of view is raised, whereas the“excessive” brightness of individual pixels is left unchanged, then the“excessively” bright pixels are not perceived as being “excessively”bright. If the information apparatus also comprises an infrared sensorthat captures infrared light from the field of view, then additional,monochrome image information with regard to the field of view can beobtained by night or fog that can be transformed into the visiblespectral range in order to improve the image information alreadyobtained via the field-of-view capturing apparatus and the evaluationapparatus.

[0229] In general, the information system in accordance with theinvention can also be suitable for improving visual acuity. E.g. in thecase of strong or weak contrasts or in the case of low brightness in thefield of view, image information that is adjusted with regard to itsbrightness can be projected into the eye in order to allow improvedvisual acuity.

[0230] The information system 100 may be integrated into a helmet suchas a fireman's helmet described above. Similar embodiments, e.g. as asoldier's, driver's, crane operator's, sportsman's or pilot's helmet orspectacles are possible. A soldier's helmet/spectacles on the basis ofthe information system in accordance with the invention could be of aidto the soldier, for example, for orientation and/or for targeting. Insuch case, the information apparatus of the information systempreferably comprises sensors and/or radio receivers that allow anextrasensory perception of the surroundings and/or the reception ofinformation from a command center. The output apparatus will provideinformation preferably visually, acoustically or tactually, e.g. in theform of short electric stimulating currents on the skin. Latter could beused to directly inform a soldier of the direction of a foreign objectapproaching from behind.

[0231] As a night vision device, the information system would alsocapture infrared light from the field of view in addition to the captureof visible light from the field of view. As described above, imageinformation can be obtained from such captured infrared light and beemployed in the enhancement of image information to be projected intothe eye.

[0232] If the information apparatus comprises e.g. a GPS receiver, thenthe helmet could project position information or orientation aids ontothe retina. Preferably, the projection of such information into the eyeis carried out similar to the projection of an electronic newspaper.This avoids a distraction of the soldier since the information appearsto be fixed in space or vis-à-vis a neutral position of the eye. Anadaptation of the image information to the background perceivedtherebehind for the sake of best possible readability also takes placevia an evaluation apparatus belonging to the information apparatus.

[0233] Although a radio transmission or other data transmission from thesoldier to a command center is generally to be avoided from strategicreasons of camouflage, a transmission of field-of-view data correlatedto the eye movements of the soldier to a command center could also bemeaningful in particular cases.

[0234] In an embodiment that is particular interesting for soldiers, theinformation apparatus comprises one or more cameras that capture imagesfrom outside the field of view. The image information obtained in thismanner is then projected onto the retina via a projection apparatus. Thesupplementary image projected onto the field-of-view image could beprojected, for example, as an image within an image as a small image inthe corner of the natural or projected field-of-view image or appear asa longitudinal strip along the bottom edge. In this case, the capture ofsignals from the eye serves, together with the capture of the field ofview, to maintain the projected images in correlation with the movementsof the eye.

[0235] For a crane operator, it is helpful to project in supplementaryimages from other perspectives into the field of view. The informationsystem includes supplementary sensors providing aid distance or weightinformation that is projected into the field of view. Such informationcan also be provided audibly or visually, e.g., upon gazing at the loadin combination with the clicking of a button. In this case, the lightdetermined from the field of view serves as a basis for the imagerecognition, whereas the signals from the eye allow a correlation of thecaptured field of view to the visual axis as described above.

[0236] The information system 100 can provide a pilot with many varioustypes of information. Via a connection to the information system of anairplane, relevant data such as flight altitude, speed or direction offlight or even an artificial horizon could be blended in to the pilot'sfield of view, for example, as described above. During landing, landingaid information could also be blended in that depict a virtual landingcorridor or indicate altitude or direction correction values. Inmilitary applications, friend/foe and targeting aid information could beprovided to the pilot. In this case, the gaze direction of the pilotplays a role both during the spatial blending in of the information aswell as during information selection. The pilot would like a flyingobject upon which he has fixed his eyes' gaze to be identified. If theidentification is carried out visually, he does not want the blending into cover any relevant areas of his field of view. In this case, dueconsideration must be given to the contrary requirements that therelevant regions of the field of view are typically imaged onto thefovea centralis but also that only those images that are projected ontothe fovea centralis are sharply imaged. Thus, an intelligent blending inmust be carried out in which the relevant regions of the field of vieware recognized, for example, via image recognition and not solely viathe orientation of the fovea centralis. In this respect, the informationsystem in accordance with the invention can also act as a sub-system tothe information system of the aircraft and provide information thereto.In this manner, e.g. information with regard to where the pilot islooking could be supplied to the aircraft information system by theinformation system in accordance with the invention and contribute thereto target capturing. In true action, the information system could locateenemy radar positions via sensors and depict their position togetherwith the associated landscape in three dimensions.

[0237] Various types of information could be provided to sportsmen viathe information system in accordance with the invention as in theexamples above. Orientation aids, speed information and/or enhancedfield-of-view information that allows better vision at dusk, at night,in rainy spray or fog could be provided, for example, by projectinginformation into the eye. A non-visual provision of the information isparticularly suitable in the case of low-content information. Similar tothe above examples, an information system in accordance with theinvention worn by a sportsman could act as a sub-system of a sportingdevice or of a vehicle.

[0238] The information system 100 can provide extrasensory perceptionusing one or more sensors, e.g. magnetic field detectors, pressuresensors, thermometers, spectral sensors, optical or acousticinterference measuring devices. In particular in the case of asuperimposition of a pictorial presentation of information obtained fromthe sensors onto the natural field of view via projection into the eye,the presentation corresponds to the needs of the person having vision.In such a case, the information system in accordance with the inventioncan appear as a component, in particular as a presentation apparatus, ofa complex measuring apparatus.

[0239] An example of such a system are spectacles equipped withsensitive magnetic sensors that is in a position to locatecurrent-conducting or metallic objects in correlation to the spectacles.If such located objects are designated, true to their position and incolor, in the natural field of view by projection that enables locating,e.g., water pipes or electric wiring running under plaster. A handymanwearing such a spectacle system would see the path of the piping orwiring as “painted on the wall”.

[0240] If a two or three dimensional array or other one ormulti-dimensional distribution of the sensors is chosen, then even e.g.highly complex vector fields or gradients could be made visible to anobserver as an image over the object or arrangement associatedtherewith. For example, an arrangement of pressure sensors around a testobject in a wind tunnel could supply pressure information that isprepared, in such a manner, via the information system in accordancewith the invention as described above and projected into the eyes of anobserver who is observing the test object through a window such that hesees the pressure gradients resulting from the test object there wherethey are present based on appropriate, colored depiction of the pressurevalues. Temperature information obtained using an infrared camera couldbe presented to a welder in his field of view such that the localsurface temperature along the work piece is recognizable.

[0241] Similarly, spectral sensors could be used to give a userinformation about exact color values or material compositions. In thiscase, it is also practical to present the determined information audiblydepending on exactly where the user is looking. In conjunction with adatabank and pattern recognition, such a system could be used, forexample, to at least approximately identify mushrooms or plants, whereinthe user, upon system request, looks at particular parts of themushroom/plant and/or turns these to face the sensors.

[0242] Having described various embodiments and implementations of thepresent invention, it should be apparent to those skilled in therelevant art that the foregoing is illustrative only and not limiting,having been presented by way of example only. There are otherembodiments or elements suitable for the above-described embodiments,described in the above-listed publications, all of which areincorporated by referece as if fully reproduced herein. The functions ofany one element may be carried out in various ways in alternativeembodiments. Also, the functions of several elements may, in alternativeembodiments, be carried out by fewer, or a single, element.

What is claimed is:
 1. An information system, comprising: an opticalsignal unit constructed and positioned to capture signals associatedwith an eye; a wireless communication unit; and an output unit,interfaced with said wireless communication unit, constructed andarranged to provide information using a correlation unit constructed tofind suitable relationship between said captured signals and additionaldata.
 2. The information system of claim 1, further including aninformation unit constructed to provide said additional data, andwherein said correlation unit is constructed to determine a presentationrelationship of said captured signals and said additional data.
 3. Theinformation system of claim 1, wherein said correlation unit isconstructed to determine said presentation relationship between saidcaptured signals and said additional data in terms of location of saidadditional data with respect of said captured data.
 4. The informationsystem of claim 1, wherein said correlation unit is constructed todetermine said presentation relationship between said captured signalsand said additional data in terms of presentation timing of saidadditional data with respect of said captured data.
 5. The informationsystem of claim 1, wherein said correlation unit is constructed todetermine said presentation relationship between said captured signalsand said additional data in terms of relative color display of saidadditional data with respect of said captured data.
 6. An informationsystem, comprising an optical signal unit constructed and positioned tocapture signals reflected back from at least one eye comprising theretina; a field-of-view capturing unit constructed and arranged tocapture light from a field of view associated with said retina withoutcapturing a retinal reflex image thereof; an information unit; awireless communication unit; and an output unit constructed to provideinformation, at least partially obtained via said communication unit, incooperation with said information unit as a function of said capturedlight and in correlation with said captured signals.
 7. An informationsystem, comprising a optical signal unit constructed and positioned tocapture signals reflected back from at least one eye comprising theretina, said optical signal unit comprising a scanning detection unitconstructed to at least partially capture a retinal reflex image of saidretina; an information unit; a wireless communication unit; and anoutput unit constructed and arranged to provide information, at leastpartially obtained via said communication unit, in cooperation with saidinformation unit as a function of said captured signals, said outputunit being not capable of projecting information onto the retina.
 8. Aninformation system, comprising an optical signal unit constructed andpositioned to capture signals reflected back from at least one eyecomprising the retina, said optical signal unit comprising a scanningdetection unit constructed to at least partially capture a retinalreflex image of said retina during a first scanning operation andcarrying out a less comprehensive capture of said retinal reflex imageduring a later scanning operation; an information unit; a wirelesscommunication unit; and an output unit constructed and arranged toprovide information, at least partially obtained via said communicationunit, in cooperation with said information unit as a function of saidcaptured signals, said output unit comprising a scanning projectiondevice constructed to project at least part of said information ontosaid retina.
 9. An information system, comprising an optical signal unitconstructed and positioned to capture signals reflected back from atleast one eye without reaching the retina; an information unit; awireless communication unit; and an output unit constructed and arrangedto provide information, at least partially obtained via saidcommunication unit in cooperation with said information unit as afunction of said captured signals, said output unit comprising ascanning projection device constructed and arranged to project at leastpart of said information onto said retina.
 10. The information system ofclaim 9 comprising a spherical or spherical-acting reflection layeroperably positionable at a location immediately anterior andsubstantially confocal to said eye, and wherein said optical signal unitis constructed to capture optical field-of-vision signals reflected offsaid spherical or spherical-acting reflection layer.
 11. The informationsystem of claim 9 comprising a field-of-view capturing unit constructedto capture visible light from a field of view associated with the retinawithout capturing a retinal reflex image thereof; and said output unitbeing suitable for providing said information in correlation with saidcaptured visible light.
 12. The information system of claim 9, whereinsaid function encompasses a pattern recognition that yields at least oneinformation key, and said information keys serve for an informationquery based on said information apparatus.
 13. An information system,comprising a signal input unit constructed and positioned to capture atleast two types of signals reflected back from at least one eye; aninformation unit; a wireless communication unit; and an output unitconstructed and arranged to providing information, at least partiallyobtained and/or provided via said communication unit, in cooperationwith said information apparatus as a function of said captured signals,said output unit comprising a scanning projection device constructed toproject at least part of said information onto the retina of said eye.14. The information system of claims 13 comprising a spherical orspherical-acting reflection layer operably positionable at a locationimmediately anterior and substantially confocal to said eye, and whereinsaid signal unit is an optical signal unit constructed to captureoptical field-of-vision signals reflected off said spherical orspherical-acting reflection layer.
 15. The information system of claim14 wherein said signal input unit includes a field-of-view capturingunit constructed to capture visible light from a field of viewassociated with the retina without capturing a retinal reflex imagethereof; and said output unit being suitable for providing saidinformation in correlation with said captured visible light.
 16. Theinformation system of claim 15, wherein said information unit comprisesan evaluation module constructed to obtain image information with regardto said field of view from said captured visible light; and saidprojection device is constructed to project the image information ontothe retina in correlation with said captured signals such that anaturally perceived field of view and projected image information areperceived as a unitary image by the retina.
 17. The information systemof claim 16 wherein said function encompasses a temporal correlationbetween said provision of information and said captured light.
 18. Theinformation system of claim 16 wherein said function encompasses aspatial correlation between said provision of information and saidcaptured light.
 19. The information system of claim 16 wherein saidfunction encompasses a pattern recognition that yields at least oneinformation key, and said information key serves for an informationquery based on said information apparatus.
 20. A method of creating andproviding information, comprising the acts of: capturing signalsassociated with an eye; enabling a wireless communication of data; andproviding information by using said wireless communication andcorrelating said captured signals and additional data.